2023-02-13 04:54:27,302 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 04:54:27,540 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655 2023-02-13 04:54:27,556 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 04:54:27,557 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea, deleteOnExit=true 2023-02-13 04:54:27,557 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 04:54:27,558 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/test.cache.data in system properties and HBase conf 2023-02-13 04:54:27,558 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 04:54:27,559 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/hadoop.log.dir in system properties and HBase conf 2023-02-13 04:54:27,560 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 04:54:27,560 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 04:54:27,560 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 04:54:27,667 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 04:54:28,041 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 04:54:28,045 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 04:54:28,045 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 04:54:28,045 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 04:54:28,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 04:54:28,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 04:54:28,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 04:54:28,046 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 04:54:28,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 04:54:28,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 04:54:28,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/nfs.dump.dir in system properties and HBase conf 2023-02-13 04:54:28,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/java.io.tmpdir in system properties and HBase conf 2023-02-13 04:54:28,048 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 04:54:28,048 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 04:54:28,048 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 04:54:28,510 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 04:54:28,515 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 04:54:28,795 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 04:54:28,943 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 04:54:28,957 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 04:54:28,997 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 04:54:29,029 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/java.io.tmpdir/Jetty_localhost_44897_hdfs____31eg74/webapp 2023-02-13 04:54:29,194 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44897 2023-02-13 04:54:29,205 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 04:54:29,205 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 04:54:29,598 WARN [Listener at localhost/44231] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 04:54:29,692 WARN [Listener at localhost/44231] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 04:54:29,710 WARN [Listener at localhost/44231] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 04:54:29,715 INFO [Listener at localhost/44231] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 04:54:29,720 INFO [Listener at localhost/44231] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/java.io.tmpdir/Jetty_localhost_46779_datanode____yt1dku/webapp 2023-02-13 04:54:29,814 INFO [Listener at localhost/44231] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46779 2023-02-13 04:54:30,117 WARN [Listener at localhost/42449] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 04:54:30,130 WARN [Listener at localhost/42449] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 04:54:30,133 WARN [Listener at localhost/42449] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 04:54:30,136 INFO [Listener at localhost/42449] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 04:54:30,142 INFO [Listener at localhost/42449] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/java.io.tmpdir/Jetty_localhost_42225_datanode____54t2qm/webapp 2023-02-13 04:54:30,251 INFO [Listener at localhost/42449] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42225 2023-02-13 04:54:30,261 WARN [Listener at localhost/36699] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 04:54:30,276 WARN [Listener at localhost/36699] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 04:54:30,279 WARN [Listener at localhost/36699] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 04:54:30,281 INFO [Listener at localhost/36699] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 04:54:30,290 INFO [Listener at localhost/36699] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/java.io.tmpdir/Jetty_localhost_33043_datanode____.bji3wq/webapp 2023-02-13 04:54:30,400 INFO [Listener at localhost/36699] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33043 2023-02-13 04:54:30,410 WARN [Listener at localhost/33551] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 04:54:30,585 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x31303dd4c0378dde: Processing first storage report for DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3 from datanode a51c43df-067e-41c1-a591-2b7ca0d868cc 2023-02-13 04:54:30,586 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x31303dd4c0378dde: from storage DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3 node DatanodeRegistration(127.0.0.1:37933, datanodeUuid=a51c43df-067e-41c1-a591-2b7ca0d868cc, infoPort=40847, infoSecurePort=0, ipcPort=42449, storageInfo=lv=-57;cid=testClusterID;nsid=168932056;c=1676264068588), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 04:54:30,586 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb4554061bf87bd6f: Processing first storage report for DS-8b2a8137-8605-4c97-bf6f-80b654149536 from datanode 2af29379-a15b-4462-998d-be0242ce1402 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb4554061bf87bd6f: from storage DS-8b2a8137-8605-4c97-bf6f-80b654149536 node DatanodeRegistration(127.0.0.1:46739, datanodeUuid=2af29379-a15b-4462-998d-be0242ce1402, infoPort=37137, infoSecurePort=0, ipcPort=36699, storageInfo=lv=-57;cid=testClusterID;nsid=168932056;c=1676264068588), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7f8ec153f47466ff: Processing first storage report for DS-a234c2db-ab31-41ba-b559-606a61b21a30 from datanode 82110bba-6b14-4557-aa3b-b536cbd539dd 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7f8ec153f47466ff: from storage DS-a234c2db-ab31-41ba-b559-606a61b21a30 node DatanodeRegistration(127.0.0.1:33533, datanodeUuid=82110bba-6b14-4557-aa3b-b536cbd539dd, infoPort=46281, infoSecurePort=0, ipcPort=33551, storageInfo=lv=-57;cid=testClusterID;nsid=168932056;c=1676264068588), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x31303dd4c0378dde: Processing first storage report for DS-a2929c6d-77bd-4684-8ea5-08efa0ae9dc2 from datanode a51c43df-067e-41c1-a591-2b7ca0d868cc 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x31303dd4c0378dde: from storage DS-a2929c6d-77bd-4684-8ea5-08efa0ae9dc2 node DatanodeRegistration(127.0.0.1:37933, datanodeUuid=a51c43df-067e-41c1-a591-2b7ca0d868cc, infoPort=40847, infoSecurePort=0, ipcPort=42449, storageInfo=lv=-57;cid=testClusterID;nsid=168932056;c=1676264068588), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb4554061bf87bd6f: Processing first storage report for DS-f5319d5a-ae04-497e-a04e-305a5d4e522f from datanode 2af29379-a15b-4462-998d-be0242ce1402 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb4554061bf87bd6f: from storage DS-f5319d5a-ae04-497e-a04e-305a5d4e522f node DatanodeRegistration(127.0.0.1:46739, datanodeUuid=2af29379-a15b-4462-998d-be0242ce1402, infoPort=37137, infoSecurePort=0, ipcPort=36699, storageInfo=lv=-57;cid=testClusterID;nsid=168932056;c=1676264068588), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7f8ec153f47466ff: Processing first storage report for DS-4edb1503-d385-4f6e-b15e-01094f6a057f from datanode 82110bba-6b14-4557-aa3b-b536cbd539dd 2023-02-13 04:54:30,587 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7f8ec153f47466ff: from storage DS-4edb1503-d385-4f6e-b15e-01094f6a057f node DatanodeRegistration(127.0.0.1:33533, datanodeUuid=82110bba-6b14-4557-aa3b-b536cbd539dd, infoPort=46281, infoSecurePort=0, ipcPort=33551, storageInfo=lv=-57;cid=testClusterID;nsid=168932056;c=1676264068588), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 04:54:30,790 DEBUG [Listener at localhost/33551] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655 2023-02-13 04:54:30,849 INFO [Listener at localhost/33551] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/zookeeper_0, clientPort=55056, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 04:54:30,865 INFO [Listener at localhost/33551] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55056 2023-02-13 04:54:30,872 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:30,874 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:31,536 INFO [Listener at localhost/33551] util.FSUtils(479): Created version file at hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb with version=8 2023-02-13 04:54:31,536 INFO [Listener at localhost/33551] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/hbase-staging 2023-02-13 04:54:31,825 INFO [Listener at localhost/33551] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 04:54:32,276 INFO [Listener at localhost/33551] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 04:54:32,307 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,308 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,309 INFO [Listener at localhost/33551] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 04:54:32,309 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,309 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 04:54:32,445 INFO [Listener at localhost/33551] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 04:54:32,514 DEBUG [Listener at localhost/33551] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 04:54:32,609 INFO [Listener at localhost/33551] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43801 2023-02-13 04:54:32,619 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,621 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,640 INFO [Listener at localhost/33551] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43801 connecting to ZooKeeper ensemble=127.0.0.1:55056 2023-02-13 04:54:32,679 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:438010x0, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 04:54:32,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43801-0x1004c07c8780000 connected 2023-02-13 04:54:32,703 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 04:54:32,704 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:54:32,707 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 04:54:32,715 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43801 2023-02-13 04:54:32,715 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43801 2023-02-13 04:54:32,716 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43801 2023-02-13 04:54:32,716 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43801 2023-02-13 04:54:32,716 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43801 2023-02-13 04:54:32,722 INFO [Listener at localhost/33551] master.HMaster(439): hbase.rootdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb, hbase.cluster.distributed=false 2023-02-13 04:54:32,788 INFO [Listener at localhost/33551] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 04:54:32,788 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,789 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,789 INFO [Listener at localhost/33551] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 04:54:32,789 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,789 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 04:54:32,793 INFO [Listener at localhost/33551] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 04:54:32,796 INFO [Listener at localhost/33551] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46601 2023-02-13 04:54:32,798 INFO [Listener at localhost/33551] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 04:54:32,804 DEBUG [Listener at localhost/33551] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 04:54:32,805 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,807 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,809 INFO [Listener at localhost/33551] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46601 connecting to ZooKeeper ensemble=127.0.0.1:55056 2023-02-13 04:54:32,812 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:466010x0, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 04:54:32,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46601-0x1004c07c8780001 connected 2023-02-13 04:54:32,813 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 04:54:32,815 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:54:32,816 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 04:54:32,816 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46601 2023-02-13 04:54:32,816 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46601 2023-02-13 04:54:32,817 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46601 2023-02-13 04:54:32,818 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46601 2023-02-13 04:54:32,818 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46601 2023-02-13 04:54:32,830 INFO [Listener at localhost/33551] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 04:54:32,830 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,830 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,831 INFO [Listener at localhost/33551] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 04:54:32,831 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,831 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 04:54:32,831 INFO [Listener at localhost/33551] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 04:54:32,833 INFO [Listener at localhost/33551] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33859 2023-02-13 04:54:32,833 INFO [Listener at localhost/33551] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 04:54:32,834 DEBUG [Listener at localhost/33551] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 04:54:32,835 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,837 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,838 INFO [Listener at localhost/33551] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33859 connecting to ZooKeeper ensemble=127.0.0.1:55056 2023-02-13 04:54:32,842 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:338590x0, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 04:54:32,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33859-0x1004c07c8780002 connected 2023-02-13 04:54:32,844 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 04:54:32,844 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:54:32,845 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 04:54:32,846 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33859 2023-02-13 04:54:32,846 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33859 2023-02-13 04:54:32,847 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33859 2023-02-13 04:54:32,848 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33859 2023-02-13 04:54:32,848 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33859 2023-02-13 04:54:32,860 INFO [Listener at localhost/33551] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 04:54:32,860 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,860 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,860 INFO [Listener at localhost/33551] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 04:54:32,860 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 04:54:32,860 INFO [Listener at localhost/33551] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 04:54:32,861 INFO [Listener at localhost/33551] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 04:54:32,862 INFO [Listener at localhost/33551] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36009 2023-02-13 04:54:32,862 INFO [Listener at localhost/33551] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 04:54:32,863 DEBUG [Listener at localhost/33551] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 04:54:32,864 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,866 INFO [Listener at localhost/33551] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:32,867 INFO [Listener at localhost/33551] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36009 connecting to ZooKeeper ensemble=127.0.0.1:55056 2023-02-13 04:54:32,871 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:360090x0, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 04:54:32,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36009-0x1004c07c8780003 connected 2023-02-13 04:54:32,872 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 04:54:32,872 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:54:32,873 DEBUG [Listener at localhost/33551] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 04:54:32,874 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36009 2023-02-13 04:54:32,874 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36009 2023-02-13 04:54:32,875 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36009 2023-02-13 04:54:32,875 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36009 2023-02-13 04:54:32,875 DEBUG [Listener at localhost/33551] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36009 2023-02-13 04:54:32,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:32,888 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 04:54:32,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:32,906 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 04:54:32,906 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 04:54:32,906 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 04:54:32,906 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 04:54:32,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:32,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 04:54:32,909 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43801,1676264071669 from backup master directory 2023-02-13 04:54:32,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 04:54:32,913 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:32,913 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 04:54:32,914 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 04:54:32,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:32,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 04:54:32,918 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 04:54:33,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/hbase.id with ID: 7f514ebe-6fbc-4d7a-b67a-aecb06b396e7 2023-02-13 04:54:33,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 04:54:33,066 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:33,117 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x02cce10f to 127.0.0.1:55056 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 04:54:33,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@61875813, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 04:54:33,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 04:54:33,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 04:54:33,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 04:54:33,195 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 04:54:33,197 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:54:33,201 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:54:33,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 04:54:33,235 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store-tmp 2023-02-13 04:54:33,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:33,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 04:54:33,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:54:33,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:54:33,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 04:54:33,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:54:33,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:54:33,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676264073283Disabling compacts and flushes for region at 1676264073283Disabling writes for close at 1676264073284 (+1 ms)Writing region close event to WAL at 1676264073284Closed at 1676264073284 2023-02-13 04:54:33,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/WALs/jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:33,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43801%2C1676264071669, suffix=, logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/WALs/jenkins-hbase4.apache.org,43801,1676264071669, archiveDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/oldWALs, maxLogs=10 2023-02-13 04:54:33,359 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK] 2023-02-13 04:54:33,359 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK] 2023-02-13 04:54:33,359 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK] 2023-02-13 04:54:33,366 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:54:33,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/WALs/jenkins-hbase4.apache.org,43801,1676264071669/jenkins-hbase4.apache.org%2C43801%2C1676264071669.1676264073316 2023-02-13 04:54:33,433 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK], DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK], DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK]] 2023-02-13 04:54:33,433 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:33,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:33,437 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 04:54:33,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 04:54:33,495 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 04:54:33,502 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 04:54:33,527 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 04:54:33,541 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:33,548 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 04:54:33,550 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 04:54:33,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 04:54:33,569 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:33,570 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64150163, jitterRate=-0.044088080525398254}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 04:54:33,571 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676264073462Initializing all the Stores at 1676264073464 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264073465 (+1 ms)Cleaning up temporary data from old regions at 1676264073557 (+92 ms)Cleaning up detritus from prior splits at 1676264073558 (+1 ms)Region opened successfully at 1676264073571 (+13 ms) 2023-02-13 04:54:33,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 04:54:33,591 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:33,591 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:33,594 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:33,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 04:54:33,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-13 04:54:33,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 04:54:33,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 04:54:33,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 04:54:33,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 04:54:33,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 04:54:33,686 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 04:54:33,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 04:54:33,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 04:54:33,698 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:33,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 04:54:33,700 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 04:54:33,716 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 04:54:33,720 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 04:54:33,720 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 04:54:33,720 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 04:54:33,720 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 04:54:33,720 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:33,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,43801,1676264071669, sessionid=0x1004c07c8780000, setting cluster-up flag (Was=false) 2023-02-13 04:54:33,736 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:33,741 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 04:54:33,742 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:33,748 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:33,753 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 04:54:33,754 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:33,757 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.hbase-snapshot/.tmp 2023-02-13 04:54:33,780 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(952): ClusterId : 7f514ebe-6fbc-4d7a-b67a-aecb06b396e7 2023-02-13 04:54:33,780 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(952): ClusterId : 7f514ebe-6fbc-4d7a-b67a-aecb06b396e7 2023-02-13 04:54:33,781 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(952): ClusterId : 7f514ebe-6fbc-4d7a-b67a-aecb06b396e7 2023-02-13 04:54:33,786 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 04:54:33,786 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 04:54:33,786 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 04:54:33,792 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 04:54:33,792 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 04:54:33,792 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 04:54:33,792 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 04:54:33,792 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 04:54:33,792 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 04:54:33,797 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 04:54:33,797 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 04:54:33,797 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 04:54:33,799 DEBUG [RS:2;jenkins-hbase4:36009] zookeeper.ReadOnlyZKClient(139): Connect 0x1c9d1aed to 127.0.0.1:55056 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 04:54:33,799 DEBUG [RS:0;jenkins-hbase4:46601] zookeeper.ReadOnlyZKClient(139): Connect 0x15e8e6b1 to 127.0.0.1:55056 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 04:54:33,799 DEBUG [RS:1;jenkins-hbase4:33859] zookeeper.ReadOnlyZKClient(139): Connect 0x5cf9bd84 to 127.0.0.1:55056 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 04:54:33,809 DEBUG [RS:2;jenkins-hbase4:36009] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@125ae70e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 04:54:33,810 DEBUG [RS:1;jenkins-hbase4:33859] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@523e586b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 04:54:33,810 DEBUG [RS:2;jenkins-hbase4:36009] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1665719d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 04:54:33,810 DEBUG [RS:1;jenkins-hbase4:33859] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5601d6f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 04:54:33,810 DEBUG [RS:0;jenkins-hbase4:46601] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b2f0a7c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 04:54:33,811 DEBUG [RS:0;jenkins-hbase4:46601] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f108dc1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 04:54:33,832 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46601 2023-02-13 04:54:33,836 INFO [RS:0;jenkins-hbase4:46601] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 04:54:33,836 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:36009 2023-02-13 04:54:33,836 INFO [RS:0;jenkins-hbase4:46601] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 04:54:33,837 INFO [RS:2;jenkins-hbase4:36009] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 04:54:33,837 INFO [RS:2;jenkins-hbase4:36009] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 04:54:33,837 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 04:54:33,838 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:33859 2023-02-13 04:54:33,837 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 04:54:33,838 INFO [RS:1;jenkins-hbase4:33859] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 04:54:33,838 INFO [RS:1;jenkins-hbase4:33859] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 04:54:33,838 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 04:54:33,840 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43801,1676264071669 with isa=jenkins-hbase4.apache.org/172.31.14.131:33859, startcode=1676264072829 2023-02-13 04:54:33,840 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43801,1676264071669 with isa=jenkins-hbase4.apache.org/172.31.14.131:36009, startcode=1676264072859 2023-02-13 04:54:33,840 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43801,1676264071669 with isa=jenkins-hbase4.apache.org/172.31.14.131:46601, startcode=1676264072787 2023-02-13 04:54:33,858 DEBUG [RS:1;jenkins-hbase4:33859] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 04:54:33,858 DEBUG [RS:0;jenkins-hbase4:46601] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 04:54:33,858 DEBUG [RS:2;jenkins-hbase4:36009] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 04:54:33,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:33,911 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40287, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 04:54:33,911 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35091, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 04:54:33,911 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35631, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 04:54:33,912 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 04:54:33,913 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:33,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264103914 2023-02-13 04:54:33,916 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 04:54:33,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 04:54:33,922 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:33,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 04:54:33,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43801] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-13 04:54:33,922 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 04:54:33,927 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 04:54:33,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 04:54:33,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 04:54:33,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 04:54:33,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 04:54:33,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 04:54:33,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:33,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 04:54:33,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 04:54:33,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 04:54:33,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 04:54:33,945 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 04:54:33,947 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264073946,5,FailOnTimeoutGroup] 2023-02-13 04:54:33,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264073947,5,FailOnTimeoutGroup] 2023-02-13 04:54:33,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:33,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 04:54:33,950 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:33,950 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:33,955 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 04:54:33,955 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 04:54:33,955 WARN [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 04:54:33,955 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(2829): Master is not running yet 2023-02-13 04:54:33,955 WARN [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 04:54:33,956 WARN [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-13 04:54:33,975 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:33,975 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:33,976 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb 2023-02-13 04:54:33,997 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:33,999 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 04:54:34,002 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/info 2023-02-13 04:54:34,002 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 04:54:34,003 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:34,003 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 04:54:34,005 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/rep_barrier 2023-02-13 04:54:34,006 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 04:54:34,007 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:34,007 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 04:54:34,009 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/table 2023-02-13 04:54:34,009 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 04:54:34,010 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:34,011 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740 2023-02-13 04:54:34,012 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740 2023-02-13 04:54:34,015 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 04:54:34,017 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 04:54:34,020 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:34,021 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65879352, jitterRate=-0.01832115650177002}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 04:54:34,021 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676264073997Initializing all the Stores at 1676264073999 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264073999Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264073999Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264073999Cleaning up temporary data from old regions at 1676264074014 (+15 ms)Cleaning up detritus from prior splits at 1676264074014Region opened successfully at 1676264074021 (+7 ms) 2023-02-13 04:54:34,021 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 04:54:34,022 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 04:54:34,022 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 04:54:34,022 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 04:54:34,022 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 04:54:34,023 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 04:54:34,023 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676264074021Disabling compacts and flushes for region at 1676264074021Disabling writes for close at 1676264074022 (+1 ms)Writing region close event to WAL at 1676264074023 (+1 ms)Closed at 1676264074023 2023-02-13 04:54:34,029 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:34,029 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 04:54:34,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 04:54:34,051 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 04:54:34,053 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 04:54:34,056 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43801,1676264071669 with isa=jenkins-hbase4.apache.org/172.31.14.131:36009, startcode=1676264072859 2023-02-13 04:54:34,057 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43801,1676264071669 with isa=jenkins-hbase4.apache.org/172.31.14.131:46601, startcode=1676264072787 2023-02-13 04:54:34,057 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43801,1676264071669 with isa=jenkins-hbase4.apache.org/172.31.14.131:33859, startcode=1676264072829 2023-02-13 04:54:34,062 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43801] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,063 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,069 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb 2023-02-13 04:54:34,069 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb 2023-02-13 04:54:34,069 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb 2023-02-13 04:54:34,069 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44231 2023-02-13 04:54:34,069 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44231 2023-02-13 04:54:34,069 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 04:54:34,069 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:44231 2023-02-13 04:54:34,069 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 04:54:34,069 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 04:54:34,083 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:54:34,084 DEBUG [RS:1;jenkins-hbase4:33859] zookeeper.ZKUtil(162): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,084 DEBUG [RS:0;jenkins-hbase4:46601] zookeeper.ZKUtil(162): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,084 DEBUG [RS:2;jenkins-hbase4:36009] zookeeper.ZKUtil(162): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,084 WARN [RS:0;jenkins-hbase4:46601] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 04:54:34,084 WARN [RS:2;jenkins-hbase4:36009] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 04:54:34,084 WARN [RS:1;jenkins-hbase4:33859] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 04:54:34,085 INFO [RS:2;jenkins-hbase4:36009] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 04:54:34,085 INFO [RS:1;jenkins-hbase4:33859] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 04:54:34,085 INFO [RS:0;jenkins-hbase4:46601] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 04:54:34,085 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,086 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,085 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1947): logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,086 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36009,1676264072859] 2023-02-13 04:54:34,086 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46601,1676264072787] 2023-02-13 04:54:34,086 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33859,1676264072829] 2023-02-13 04:54:34,104 DEBUG [RS:1;jenkins-hbase4:33859] zookeeper.ZKUtil(162): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,104 DEBUG [RS:0;jenkins-hbase4:46601] zookeeper.ZKUtil(162): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,104 DEBUG [RS:2;jenkins-hbase4:36009] zookeeper.ZKUtil(162): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,104 DEBUG [RS:1;jenkins-hbase4:33859] zookeeper.ZKUtil(162): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,104 DEBUG [RS:0;jenkins-hbase4:46601] zookeeper.ZKUtil(162): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,104 DEBUG [RS:2;jenkins-hbase4:36009] zookeeper.ZKUtil(162): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,105 DEBUG [RS:1;jenkins-hbase4:33859] zookeeper.ZKUtil(162): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,105 DEBUG [RS:0;jenkins-hbase4:46601] zookeeper.ZKUtil(162): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,105 DEBUG [RS:2;jenkins-hbase4:36009] zookeeper.ZKUtil(162): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,114 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 04:54:34,114 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 04:54:34,114 DEBUG [RS:0;jenkins-hbase4:46601] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 04:54:34,123 INFO [RS:2;jenkins-hbase4:36009] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 04:54:34,123 INFO [RS:0;jenkins-hbase4:46601] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 04:54:34,123 INFO [RS:1;jenkins-hbase4:33859] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 04:54:34,143 INFO [RS:0;jenkins-hbase4:46601] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 04:54:34,143 INFO [RS:1;jenkins-hbase4:33859] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 04:54:34,144 INFO [RS:2;jenkins-hbase4:36009] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 04:54:34,147 INFO [RS:1;jenkins-hbase4:33859] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 04:54:34,147 INFO [RS:2;jenkins-hbase4:36009] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 04:54:34,147 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,147 INFO [RS:0;jenkins-hbase4:46601] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 04:54:34,147 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,148 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,148 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 04:54:34,148 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 04:54:34,148 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 04:54:34,155 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,155 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,155 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,156 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,156 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,156 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,156 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,156 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 04:54:34,157 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,158 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,157 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,158 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,158 DEBUG [RS:1;jenkins-hbase4:33859] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,158 DEBUG [RS:0;jenkins-hbase4:46601] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,158 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 04:54:34,158 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,158 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,159 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,159 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,159 DEBUG [RS:2;jenkins-hbase4:36009] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 04:54:34,159 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,159 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,160 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,160 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,160 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,161 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,161 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,161 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,176 INFO [RS:1;jenkins-hbase4:33859] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 04:54:34,176 INFO [RS:0;jenkins-hbase4:46601] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 04:54:34,176 INFO [RS:2;jenkins-hbase4:36009] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 04:54:34,178 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33859,1676264072829-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,178 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36009,1676264072859-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,178 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46601,1676264072787-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,194 INFO [RS:1;jenkins-hbase4:33859] regionserver.Replication(203): jenkins-hbase4.apache.org,33859,1676264072829 started 2023-02-13 04:54:34,195 INFO [RS:2;jenkins-hbase4:36009] regionserver.Replication(203): jenkins-hbase4.apache.org,36009,1676264072859 started 2023-02-13 04:54:34,195 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33859,1676264072829, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33859, sessionid=0x1004c07c8780002 2023-02-13 04:54:34,195 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36009,1676264072859, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36009, sessionid=0x1004c07c8780003 2023-02-13 04:54:34,195 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 04:54:34,195 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 04:54:34,195 DEBUG [RS:1;jenkins-hbase4:33859] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,195 DEBUG [RS:2;jenkins-hbase4:36009] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,196 DEBUG [RS:1;jenkins-hbase4:33859] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33859,1676264072829' 2023-02-13 04:54:34,196 DEBUG [RS:2;jenkins-hbase4:36009] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36009,1676264072859' 2023-02-13 04:54:34,196 DEBUG [RS:2;jenkins-hbase4:36009] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:34,196 DEBUG [RS:1;jenkins-hbase4:33859] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:34,197 DEBUG [RS:2;jenkins-hbase4:36009] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:34,197 DEBUG [RS:1;jenkins-hbase4:33859] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:34,197 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 04:54:34,197 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 04:54:34,197 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 04:54:34,197 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 04:54:34,197 DEBUG [RS:2;jenkins-hbase4:36009] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:34,198 DEBUG [RS:1;jenkins-hbase4:33859] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,198 DEBUG [RS:1;jenkins-hbase4:33859] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33859,1676264072829' 2023-02-13 04:54:34,198 DEBUG [RS:1;jenkins-hbase4:33859] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 04:54:34,198 DEBUG [RS:2;jenkins-hbase4:36009] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36009,1676264072859' 2023-02-13 04:54:34,198 DEBUG [RS:2;jenkins-hbase4:36009] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 04:54:34,198 DEBUG [RS:1;jenkins-hbase4:33859] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 04:54:34,199 DEBUG [RS:2;jenkins-hbase4:36009] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 04:54:34,199 DEBUG [RS:1;jenkins-hbase4:33859] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 04:54:34,199 INFO [RS:1;jenkins-hbase4:33859] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 04:54:34,199 DEBUG [RS:2;jenkins-hbase4:36009] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 04:54:34,199 INFO [RS:1;jenkins-hbase4:33859] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 04:54:34,199 INFO [RS:2;jenkins-hbase4:36009] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 04:54:34,202 INFO [RS:0;jenkins-hbase4:46601] regionserver.Replication(203): jenkins-hbase4.apache.org,46601,1676264072787 started 2023-02-13 04:54:34,202 INFO [RS:2;jenkins-hbase4:36009] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 04:54:34,202 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,46601,1676264072787, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46601, sessionid=0x1004c07c8780001 2023-02-13 04:54:34,203 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 04:54:34,203 DEBUG [RS:0;jenkins-hbase4:46601] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,203 DEBUG [RS:0;jenkins-hbase4:46601] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46601,1676264072787' 2023-02-13 04:54:34,203 DEBUG [RS:0;jenkins-hbase4:46601] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:34,203 DEBUG [RS:0;jenkins-hbase4:46601] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:34,204 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 04:54:34,204 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 04:54:34,204 DEBUG [RS:0;jenkins-hbase4:46601] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:34,204 DEBUG [RS:0;jenkins-hbase4:46601] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46601,1676264072787' 2023-02-13 04:54:34,204 DEBUG [RS:0;jenkins-hbase4:46601] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 04:54:34,204 DEBUG [RS:0;jenkins-hbase4:46601] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 04:54:34,205 DEBUG [RS:0;jenkins-hbase4:46601] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 04:54:34,205 INFO [RS:0;jenkins-hbase4:46601] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 04:54:34,205 INFO [RS:0;jenkins-hbase4:46601] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 04:54:34,205 DEBUG [jenkins-hbase4:43801] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 04:54:34,209 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:34,214 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:34,214 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:34,214 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:34,214 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:34,217 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33859,1676264072829, state=OPENING 2023-02-13 04:54:34,224 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 04:54:34,225 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:34,226 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 04:54:34,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:34,310 INFO [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33859%2C1676264072829, suffix=, logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,33859,1676264072829, archiveDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs, maxLogs=32 2023-02-13 04:54:34,311 INFO [RS:0;jenkins-hbase4:46601] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46601%2C1676264072787, suffix=, logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,46601,1676264072787, archiveDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs, maxLogs=32 2023-02-13 04:54:34,310 INFO [RS:2;jenkins-hbase4:36009] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36009%2C1676264072859, suffix=, logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,36009,1676264072859, archiveDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs, maxLogs=32 2023-02-13 04:54:34,340 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK] 2023-02-13 04:54:34,340 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK] 2023-02-13 04:54:34,340 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK] 2023-02-13 04:54:34,340 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK] 2023-02-13 04:54:34,341 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK] 2023-02-13 04:54:34,345 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK] 2023-02-13 04:54:34,346 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK] 2023-02-13 04:54:34,346 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK] 2023-02-13 04:54:34,347 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK] 2023-02-13 04:54:34,353 INFO [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,33859,1676264072829/jenkins-hbase4.apache.org%2C33859%2C1676264072829.1676264074316 2023-02-13 04:54:34,357 DEBUG [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK], DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK], DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK]] 2023-02-13 04:54:34,358 INFO [RS:0;jenkins-hbase4:46601] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,46601,1676264072787/jenkins-hbase4.apache.org%2C46601%2C1676264072787.1676264074316 2023-02-13 04:54:34,358 DEBUG [RS:0;jenkins-hbase4:46601] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK], DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK], DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK]] 2023-02-13 04:54:34,359 INFO [RS:2;jenkins-hbase4:36009] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,36009,1676264072859/jenkins-hbase4.apache.org%2C36009%2C1676264072859.1676264074316 2023-02-13 04:54:34,359 DEBUG [RS:2;jenkins-hbase4:36009] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK], DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK], DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK]] 2023-02-13 04:54:34,411 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:34,414 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 04:54:34,417 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50484, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 04:54:34,428 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 04:54:34,428 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 04:54:34,432 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33859%2C1676264072829.meta, suffix=.meta, logDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,33859,1676264072829, archiveDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs, maxLogs=32 2023-02-13 04:54:34,452 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK] 2023-02-13 04:54:34,454 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK] 2023-02-13 04:54:34,454 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK] 2023-02-13 04:54:34,459 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/WALs/jenkins-hbase4.apache.org,33859,1676264072829/jenkins-hbase4.apache.org%2C33859%2C1676264072829.meta.1676264074433.meta 2023-02-13 04:54:34,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37933,DS-08b6a681-9821-4f5a-9a1b-26f50f9c20b3,DISK], DatanodeInfoWithStorage[127.0.0.1:33533,DS-a234c2db-ab31-41ba-b559-606a61b21a30,DISK], DatanodeInfoWithStorage[127.0.0.1:46739,DS-8b2a8137-8605-4c97-bf6f-80b654149536,DISK]] 2023-02-13 04:54:34,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:34,465 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:34,465 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 04:54:34,485 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 04:54:34,488 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 04:54:34,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 04:54:34,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:34,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 04:54:34,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 04:54:34,495 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 04:54:34,497 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/info 2023-02-13 04:54:34,497 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/info 2023-02-13 04:54:34,498 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 04:54:34,499 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:34,499 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 04:54:34,501 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/rep_barrier 2023-02-13 04:54:34,501 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/rep_barrier 2023-02-13 04:54:34,501 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 04:54:34,502 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:34,503 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 04:54:34,504 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/table 2023-02-13 04:54:34,504 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/table 2023-02-13 04:54:34,505 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 04:54:34,506 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:34,508 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740 2023-02-13 04:54:34,511 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740 2023-02-13 04:54:34,514 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 04:54:34,517 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 04:54:34,518 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73448593, jitterRate=0.09446932375431061}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 04:54:34,518 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676264074493Writing region info on filesystem at 1676264074493Initializing all the Stores at 1676264074495 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264074495Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264074495Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264074495Cleaning up temporary data from old regions at 1676264074512 (+17 ms)Cleaning up detritus from prior splits at 1676264074513 (+1 ms)Running coprocessor post-open hooks at 1676264074518 (+5 ms)Region opened successfully at 1676264074518 2023-02-13 04:54:34,529 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676264074403 2023-02-13 04:54:34,547 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 04:54:34,548 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 04:54:34,548 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33859,1676264072829, state=OPEN 2023-02-13 04:54:34,551 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 04:54:34,551 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 04:54:34,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 04:54:34,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33859,1676264072829 in 323 msec 2023-02-13 04:54:34,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 04:54:34,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 519 msec 2023-02-13 04:54:34,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 752 msec 2023-02-13 04:54:34,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676264072903, completionTime=-1 2023-02-13 04:54:34,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 04:54:34,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:34,634 DEBUG [hconnection-0x6fc298c7-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 04:54:34,637 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50500, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 04:54:34,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:34,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264134658 2023-02-13 04:54:34,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264194658 2023-02-13 04:54:34,658 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 91 msec 2023-02-13 04:54:34,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43801,1676264071669-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,691 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43801,1676264071669-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43801,1676264071669-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43801, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 04:54:34,701 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 04:54:34,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 04:54:34,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 04:54:34,721 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:34,724 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:54:34,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:54:34,751 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:34,754 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853 empty. 2023-02-13 04:54:34,754 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:34,755 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 04:54:34,794 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:34,797 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => dc413a9db9894ddca28879b9c07ec853, NAME => 'hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:54:34,819 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:34,819 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing dc413a9db9894ddca28879b9c07ec853, disabling compactions & flushes 2023-02-13 04:54:34,819 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:34,819 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:34,820 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. after waiting 0 ms 2023-02-13 04:54:34,820 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:34,820 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:34,820 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for dc413a9db9894ddca28879b9c07ec853: Waiting for close lock at 1676264074819Disabling compacts and flushes for region at 1676264074819Disabling writes for close at 1676264074820 (+1 ms)Writing region close event to WAL at 1676264074820Closed at 1676264074820 2023-02-13 04:54:34,824 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:54:34,839 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264074827"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264074827"}]},"ts":"1676264074827"} 2023-02-13 04:54:34,867 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:54:34,870 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:54:34,875 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264074870"}]},"ts":"1676264074870"} 2023-02-13 04:54:34,882 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 04:54:34,887 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:34,888 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:34,888 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:34,888 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:34,888 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:34,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=dc413a9db9894ddca28879b9c07ec853, ASSIGN}] 2023-02-13 04:54:34,896 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=dc413a9db9894ddca28879b9c07ec853, ASSIGN 2023-02-13 04:54:34,898 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=dc413a9db9894ddca28879b9c07ec853, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:35,051 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:35,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=dc413a9db9894ddca28879b9c07ec853, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:35,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264075052"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264075052"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264075052"}]},"ts":"1676264075052"} 2023-02-13 04:54:35,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure dc413a9db9894ddca28879b9c07ec853, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:35,211 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:35,211 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 04:54:35,214 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41270, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 04:54:35,220 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:35,220 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dc413a9db9894ddca28879b9c07ec853, NAME => 'hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:35,221 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:35,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:35,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,222 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,224 INFO [StoreOpener-dc413a9db9894ddca28879b9c07ec853-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,227 DEBUG [StoreOpener-dc413a9db9894ddca28879b9c07ec853-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/info 2023-02-13 04:54:35,227 DEBUG [StoreOpener-dc413a9db9894ddca28879b9c07ec853-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/info 2023-02-13 04:54:35,227 INFO [StoreOpener-dc413a9db9894ddca28879b9c07ec853-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc413a9db9894ddca28879b9c07ec853 columnFamilyName info 2023-02-13 04:54:35,228 INFO [StoreOpener-dc413a9db9894ddca28879b9c07ec853-1] regionserver.HStore(310): Store=dc413a9db9894ddca28879b9c07ec853/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:35,230 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,231 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,236 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:54:35,240 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:35,241 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened dc413a9db9894ddca28879b9c07ec853; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74886474, jitterRate=0.11589542031288147}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 04:54:35,241 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for dc413a9db9894ddca28879b9c07ec853: Running coprocessor pre-open hook at 1676264075222Writing region info on filesystem at 1676264075222Initializing all the Stores at 1676264075224 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676264075224Cleaning up temporary data from old regions at 1676264075232 (+8 ms)Cleaning up detritus from prior splits at 1676264075232Running coprocessor post-open hooks at 1676264075241 (+9 ms)Region opened successfully at 1676264075241 2023-02-13 04:54:35,243 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853., pid=6, masterSystemTime=1676264075211 2023-02-13 04:54:35,248 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:35,249 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:54:35,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:35,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676264075249"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264075249"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264075249"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264075249"}]},"ts":"1676264075249"} 2023-02-13 04:54:35,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 04:54:35,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure dc413a9db9894ddca28879b9c07ec853, server=jenkins-hbase4.apache.org,36009,1676264072859 in 197 msec 2023-02-13 04:54:35,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 04:54:35,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=dc413a9db9894ddca28879b9c07ec853, ASSIGN in 367 msec 2023-02-13 04:54:35,263 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:54:35,263 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264075263"}]},"ts":"1676264075263"} 2023-02-13 04:54:35,266 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 04:54:35,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:54:35,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 557 msec 2023-02-13 04:54:35,325 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 04:54:35,327 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 04:54:35,327 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:35,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 04:54:35,351 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41284, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 04:54:35,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:35,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 04:54:35,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-02-13 04:54:35,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:35,412 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 04:54:35,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-02-13 04:54:35,425 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 04:54:35,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 04:54:35,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.515sec 2023-02-13 04:54:35,432 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 04:54:35,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 04:54:35,434 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 04:54:35,435 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43801,1676264071669-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 04:54:35,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43801,1676264071669-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 04:54:35,445 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 04:54:35,487 DEBUG [Listener at localhost/33551] zookeeper.ReadOnlyZKClient(139): Connect 0x0c8d5aec to 127.0.0.1:55056 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 04:54:35,492 DEBUG [Listener at localhost/33551] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ff608fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 04:54:35,510 DEBUG [hconnection-0x4d838b63-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 04:54:35,523 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50508, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 04:54:35,533 INFO [Listener at localhost/33551] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:54:35,567 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=357, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=168, AvailableMemoryMB=4038 2023-02-13 04:54:35,573 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:35,582 DEBUG [Listener at localhost/33551] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 04:54:35,586 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53656, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 04:54:35,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:54:35,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:54:35,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:54:35,614 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:35,617 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:54:35,619 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:54:35,622 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:35,623 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 empty. 2023-02-13 04:54:35,626 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:35,626 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 04:54:35,706 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:35,708 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3480cf0bddc2adc7fa210c7f8e2c70b4, NAME => 'testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:54:35,726 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:35,726 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 3480cf0bddc2adc7fa210c7f8e2c70b4, disabling compactions & flushes 2023-02-13 04:54:35,726 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:35,726 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:35,726 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. after waiting 0 ms 2023-02-13 04:54:35,726 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:35,727 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:35,727 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: Waiting for close lock at 1676264075726Disabling compacts and flushes for region at 1676264075726Disabling writes for close at 1676264075726Writing region close event to WAL at 1676264075727 (+1 ms)Closed at 1676264075727 2023-02-13 04:54:35,731 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:54:35,734 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676264075733"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264075733"}]},"ts":"1676264075733"} 2023-02-13 04:54:35,736 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:54:35,738 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:54:35,738 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264075738"}]},"ts":"1676264075738"} 2023-02-13 04:54:35,741 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 04:54:35,746 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:35,748 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:35,748 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:35,748 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:35,748 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:35,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, ASSIGN}] 2023-02-13 04:54:35,752 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, ASSIGN 2023-02-13 04:54:35,754 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=false 2023-02-13 04:54:35,904 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:35,905 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=3480cf0bddc2adc7fa210c7f8e2c70b4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:35,906 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676264075905"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264075905"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264075905"}]},"ts":"1676264075905"} 2023-02-13 04:54:35,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 3480cf0bddc2adc7fa210c7f8e2c70b4, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:36,062 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,063 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 04:54:36,066 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58124, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 04:54:36,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3480cf0bddc2adc7fa210c7f8e2c70b4, NAME => 'testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:36,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:36,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:36,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,074 INFO [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,077 DEBUG [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1 2023-02-13 04:54:36,077 DEBUG [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1 2023-02-13 04:54:36,077 INFO [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3480cf0bddc2adc7fa210c7f8e2c70b4 columnFamilyName cf1 2023-02-13 04:54:36,078 INFO [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] regionserver.HStore(310): Store=3480cf0bddc2adc7fa210c7f8e2c70b4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:36,078 INFO [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,080 DEBUG [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2 2023-02-13 04:54:36,080 DEBUG [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2 2023-02-13 04:54:36,081 INFO [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3480cf0bddc2adc7fa210c7f8e2c70b4 columnFamilyName cf2 2023-02-13 04:54:36,082 INFO [StoreOpener-3480cf0bddc2adc7fa210c7f8e2c70b4-1] regionserver.HStore(310): Store=3480cf0bddc2adc7fa210c7f8e2c70b4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:36,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:36,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:36,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:36,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3480cf0bddc2adc7fa210c7f8e2c70b4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75298909, jitterRate=0.12204118072986603}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:36,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: Running coprocessor pre-open hook at 1676264076073Writing region info on filesystem at 1676264076073Initializing all the Stores at 1676264076074 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264076074Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264076074Cleaning up temporary data from old regions at 1676264076086 (+12 ms)Cleaning up detritus from prior splits at 1676264076086Running coprocessor post-open hooks at 1676264076094 (+8 ms)Region opened successfully at 1676264076094 2023-02-13 04:54:36,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4., pid=11, masterSystemTime=1676264076062 2023-02-13 04:54:36,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,101 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=3480cf0bddc2adc7fa210c7f8e2c70b4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,102 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676264076101"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264076101"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264076101"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264076101"}]},"ts":"1676264076101"} 2023-02-13 04:54:36,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 04:54:36,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 3480cf0bddc2adc7fa210c7f8e2c70b4, server=jenkins-hbase4.apache.org,46601,1676264072787 in 195 msec 2023-02-13 04:54:36,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 04:54:36,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, ASSIGN in 360 msec 2023-02-13 04:54:36,112 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:54:36,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264076112"}]},"ts":"1676264076112"} 2023-02-13 04:54:36,115 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 04:54:36,118 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:54:36,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 511 msec 2023-02-13 04:54:36,369 DEBUG [Listener at localhost/33551] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 04:54:36,373 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58126, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 04:54:36,394 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:36,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,402 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:36,402 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 04:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 04:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,403 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,405 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,405 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,405 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,405 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,406 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,406 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,409 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:36,409 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:36,410 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:36,410 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,410 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:36,411 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:36,411 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:36,411 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:36,411 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:36,410 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,411 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:36,411 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:36,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:36,413 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,413 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:36,413 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. started... 2023-02-13 04:54:36,414 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:54:36,414 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,415 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,415 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,415 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,415 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:36,415 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:36,489 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/43f598b647bf45209897fb6ac4891a3d 2023-02-13 04:54:36,548 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/58376666f832426e96c726ba640dc028 2023-02-13 04:54:36,558 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/43f598b647bf45209897fb6ac4891a3d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d 2023-02-13 04:54:36,568 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:36,571 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/58376666f832426e96c726ba640dc028 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028 2023-02-13 04:54:36,581 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:36,584 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 170ms, sequenceid=6, compaction requested=false 2023-02-13 04:54:36,585 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:36,588 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: 2023-02-13 04:54:36,588 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,588 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:36,589 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:36,589 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,589 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:36,589 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:36,592 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,592 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:36,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:36,593 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,593 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:36,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:36,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:36,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:36,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 04:54:36,600 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:36,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a895dee[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:36,600 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:36,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:36,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:36,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:36,606 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:36,608 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:36,608 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:36,609 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,609 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,609 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:36,609 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:36,610 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:36,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:36,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:36,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:36,615 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 04:54:36,615 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:36,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:36,615 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:36,616 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,616 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:36,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:36,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,622 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,622 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,622 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,623 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,623 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:36,626 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,627 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,627 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,638 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,638 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,638 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,638 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,638 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,638 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,640 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,640 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,640 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,640 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,640 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,640 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:36,641 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:36,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,643 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:36,643 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:36,743 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:36,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:36,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:36,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:36,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,776 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:36,776 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,777 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 04:54:36,777 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 04:54:36,778 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,778 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,779 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,779 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,779 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,779 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,780 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,780 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:36,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:36,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:36,782 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:36,783 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:36,783 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:36,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,783 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:36,784 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:36,785 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,785 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. started... 2023-02-13 04:54:36,785 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:36,786 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,789 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:36,805 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/51c8f9134b0148189f7596cd45ded016 2023-02-13 04:54:36,830 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/d6daeb1e1e9f4821b538f838c5727101 2023-02-13 04:54:36,839 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/51c8f9134b0148189f7596cd45ded016 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016 2023-02-13 04:54:36,846 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:36,847 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/d6daeb1e1e9f4821b538f838c5727101 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101 2023-02-13 04:54:36,856 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:36,857 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 72ms, sequenceid=19, compaction requested=false 2023-02-13 04:54:36,857 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: 2023-02-13 04:54:36,857 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:36,857 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:36,858 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:36,858 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,858 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:36,858 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:36,860 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,860 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:36,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:36,861 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,861 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:36,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:36,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:36,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 04:54:36,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a81cdef[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:36,863 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:36,863 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:36,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:36,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:36,865 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:36,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:36,865 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:36,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:36,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:36,868 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:36,868 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:36,868 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,868 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:36,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:36,868 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,868 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:36,869 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:36,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,870 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:36,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:36,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:36,875 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 04:54:36,875 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:36,875 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:36,875 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,875 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:36,877 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,877 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:36,878 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,877 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,879 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:36,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,880 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,880 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:36,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,883 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:36,884 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,884 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,893 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:36,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,894 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:36,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:36,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:36,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,894 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:36,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,895 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:36,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:36,995 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:36,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:37,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:37,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:37,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,027 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,027 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,028 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 04:54:37,028 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 04:54:37,028 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,028 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,030 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,030 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,030 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,032 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,032 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,032 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,034 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,034 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,034 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,035 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:37,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:37,035 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,035 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. started... 2023-02-13 04:54:37,035 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:37,036 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,039 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,039 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,039 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,040 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,040 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,055 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/152faa931d504454bc3df0dcf55a668a 2023-02-13 04:54:37,082 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 2023-02-13 04:54:37,091 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/152faa931d504454bc3df0dcf55a668a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a 2023-02-13 04:54:37,098 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:37,100 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 2023-02-13 04:54:37,107 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:37,109 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 74ms, sequenceid=32, compaction requested=true 2023-02-13 04:54:37,109 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: 2023-02-13 04:54:37,109 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,109 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,112 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,113 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,114 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,114 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 04:54:37,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1865d3db[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:37,117 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:37,117 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,118 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,118 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,118 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,119 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,119 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,119 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:37,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,123 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,123 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:37,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:37,128 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 04:54:37,128 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:37,128 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:37,128 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,129 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,130 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,131 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,131 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,133 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,135 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,136 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,136 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,145 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:37,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,144 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,144 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,145 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:37,145 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,146 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:37,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,146 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,146 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,147 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,147 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,147 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,147 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,147 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,147 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,246 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:37,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:37,270 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:37,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:37,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,273 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,273 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,273 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 04:54:37,274 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 04:54:37,274 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,274 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,276 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,276 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,276 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,276 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,276 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,276 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,278 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,278 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,278 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,278 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,278 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,278 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,279 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,279 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,279 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,279 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,280 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,280 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,280 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. started... 2023-02-13 04:54:37,281 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:37,283 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,284 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,284 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,284 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,284 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,284 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,302 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/c2977b24e41a4e4983bc3540b82d32f0 2023-02-13 04:54:37,327 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 2023-02-13 04:54:37,336 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/c2977b24e41a4e4983bc3540b82d32f0 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0 2023-02-13 04:54:37,343 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:37,344 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 2023-02-13 04:54:37,353 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:37,354 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 73ms, sequenceid=45, compaction requested=true 2023-02-13 04:54:37,354 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: 2023-02-13 04:54:37,354 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,354 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:37,355 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:37,355 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,355 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,355 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,359 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,359 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,360 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,360 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 04:54:37,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25699ce0[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:37,362 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:37,362 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:37,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:37,365 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:37,367 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,367 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,367 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,368 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,368 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,368 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,368 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,368 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,368 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,368 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:37,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:37,372 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 04:54:37,372 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:37,372 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:37,372 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,372 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,375 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,375 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,376 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,378 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,378 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,379 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:37,384 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,384 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,384 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,384 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,385 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,385 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:37,385 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,385 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,385 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:37,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,385 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,386 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,386 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,386 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,486 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:37,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:37,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:37,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:37,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,511 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,512 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 04:54:37,512 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 04:54:37,512 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,512 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,513 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,513 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,514 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,514 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,514 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,514 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:37,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,515 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,516 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,516 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,518 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,518 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,518 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,518 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,519 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,519 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,518 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,519 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,519 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,519 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,521 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,521 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:37,521 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:37,521 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,521 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. started... 2023-02-13 04:54:37,521 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:37,522 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,522 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,523 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,541 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/150a0082b404460795717fa538d7d88b 2023-02-13 04:54:37,567 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 2023-02-13 04:54:37,574 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/150a0082b404460795717fa538d7d88b as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b 2023-02-13 04:54:37,581 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:37,582 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 2023-02-13 04:54:37,589 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:37,590 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 69ms, sequenceid=58, compaction requested=true 2023-02-13 04:54:37,590 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: 2023-02-13 04:54:37,590 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,591 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:37,591 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:37,591 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,591 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,591 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,593 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,594 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,594 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 04:54:37,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@755fd38e[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:37,596 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:37,596 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:37,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,598 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:37,598 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:37,600 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,600 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,600 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,601 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,601 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,602 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:37,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:37,607 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 04:54:37,607 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:37,607 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:37,607 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,607 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,609 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,610 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,610 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,612 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,613 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,613 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,620 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,620 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,620 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:37,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:37,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,620 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,721 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:37,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:37,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:37,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:37,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,745 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,745 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,746 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 04:54:37,746 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 04:54:37,747 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,747 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,748 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,748 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,748 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,748 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,748 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,748 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,750 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 04:54:37,751 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,751 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,751 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,751 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 04:54:37,751 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,751 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:37,752 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,751 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:37,752 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. started... 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,752 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,753 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:37,754 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,754 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,754 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,755 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,755 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,755 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,770 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/ed6e91d74655464b91bf13df070efd7f 2023-02-13 04:54:37,795 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 2023-02-13 04:54:37,803 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/ed6e91d74655464b91bf13df070efd7f as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f 2023-02-13 04:54:37,811 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:37,812 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 2023-02-13 04:54:37,820 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:37,822 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 70ms, sequenceid=71, compaction requested=true 2023-02-13 04:54:37,822 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: 2023-02-13 04:54:37,822 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:37,822 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,824 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 04:54:37,824 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 04:54:37,825 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,825 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,826 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,826 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:37,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 04:54:37,828 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1155867f[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:37,828 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:37,828 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,829 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,829 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,829 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,830 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 04:54:37,830 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:37,832 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,832 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,832 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,833 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,833 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,834 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,833 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 04:54:37,834 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,834 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,834 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 04:54:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:37,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:37,838 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 04:54:37,838 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:37,838 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:37,838 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,838 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:37,841 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,842 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,842 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:37,844 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,845 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,845 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,848 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:37,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 04:54:37,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:37,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,849 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:37,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,849 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:37,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,850 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 04:54:37,950 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:37,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:37,974 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,980 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:37,981 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:37,981 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:37,982 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:37,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, UNASSIGN}] 2023-02-13 04:54:37,990 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, UNASSIGN 2023-02-13 04:54:37,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=3480cf0bddc2adc7fa210c7f8e2c70b4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:37,992 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676264077991"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264077991"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264077991"}]},"ts":"1676264077991"} 2023-02-13 04:54:37,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 3480cf0bddc2adc7fa210c7f8e2c70b4, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:38,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3480cf0bddc2adc7fa210c7f8e2c70b4, disabling compactions & flushes 2023-02-13 04:54:38,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:38,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:38,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. after waiting 0 ms 2023-02-13 04:54:38,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:38,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:54:38,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/5b3e423638f8481fad9715a1a35a99f7 2023-02-13 04:54:38,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 2023-02-13 04:54:38,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf1/5b3e423638f8481fad9715a1a35a99f7 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7 2023-02-13 04:54:38,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:54:38,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.tmp/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 2023-02-13 04:54:38,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:54:38,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 64ms, sequenceid=82, compaction requested=true 2023-02-13 04:54:38,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:54:38,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:38,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. 2023-02-13 04:54:38,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3480cf0bddc2adc7fa210c7f8e2c70b4: Waiting for close lock at 1676264078155Running coprocessor pre-close hooks at 1676264078155Disabling compacts and flushes for region at 1676264078155Disabling writes for close at 1676264078155Obtaining lock to block concurrent updates at 1676264078155Preparing flush snapshotting stores in 3480cf0bddc2adc7fa210c7f8e2c70b4 at 1676264078155Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676264078155Flushing stores of testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. at 1676264078156 (+1 ms)Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf1: creating writer at 1676264078157 (+1 ms)Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf1: appending metadata at 1676264078160 (+3 ms)Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf1: closing flushed file at 1676264078160Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf2: creating writer at 1676264078181 (+21 ms)Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf2: appending metadata at 1676264078183 (+2 ms)Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf2: closing flushed file at 1676264078183Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf1: reopening flushed file at 1676264078202 (+19 ms)Flushing 3480cf0bddc2adc7fa210c7f8e2c70b4/cf2: reopening flushed file at 1676264078213 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3480cf0bddc2adc7fa210c7f8e2c70b4 in 64ms, sequenceid=82, compaction requested=true at 1676264078219 (+6 ms)Writing region close event to WAL at 1676264078235 (+16 ms)Running coprocessor post-close hooks at 1676264078241 (+6 ms)Closed at 1676264078241 2023-02-13 04:54:38,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,244 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=3480cf0bddc2adc7fa210c7f8e2c70b4, regionState=CLOSED 2023-02-13 04:54:38,245 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676264078244"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264078244"}]},"ts":"1676264078244"} 2023-02-13 04:54:38,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 04:54:38,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 3480cf0bddc2adc7fa210c7f8e2c70b4, server=jenkins-hbase4.apache.org,46601,1676264072787 in 253 msec 2023-02-13 04:54:38,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 04:54:38,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=3480cf0bddc2adc7fa210c7f8e2c70b4, UNASSIGN in 261 msec 2023-02-13 04:54:38,271 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=3480cf0bddc2adc7fa210c7f8e2c70b4, threads=14 2023-02-13 04:54:38,272 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,272 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,272 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,272 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,272 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,272 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,273 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,273 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,273 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,278 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,278 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,278 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,278 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,279 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,306 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3, top=true 2023-02-13 04:54:38,308 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3, top=true 2023-02-13 04:54:38,310 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b, top=true 2023-02-13 04:54:38,311 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7, top=true 2023-02-13 04:54:38,311 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4, top=true 2023-02-13 04:54:38,313 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f, top=true 2023-02-13 04:54:38,370 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.splits/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7 for child: e5e5be97dcb4e9a93e2c911fecc4d10b, parent: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,370 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.splits/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f for child: e5e5be97dcb4e9a93e2c911fecc4d10b, parent: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,371 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,370 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,371 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.splits/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4 for child: e5e5be97dcb4e9a93e2c911fecc4d10b, parent: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,371 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,372 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.splits/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3 for child: e5e5be97dcb4e9a93e2c911fecc4d10b, parent: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,372 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,376 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.splits/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3 for child: e5e5be97dcb4e9a93e2c911fecc4d10b, parent: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,376 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/.splits/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b for child: e5e5be97dcb4e9a93e2c911fecc4d10b, parent: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,376 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,376 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,472 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,487 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,493 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,500 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,506 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,511 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,511 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,513 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 for region: 3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:38,513 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 3480cf0bddc2adc7fa210c7f8e2c70b4 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:54:38,563 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:54:38,566 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:54:38,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676264078568"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676264078568"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676264078568"}]},"ts":"1676264078568"} 2023-02-13 04:54:38,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264078568"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264078568"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264078568"}]},"ts":"1676264078568"} 2023-02-13 04:54:38,569 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264078568"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264078568"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264078568"}]},"ts":"1676264078568"} 2023-02-13 04:54:38,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, ASSIGN}] 2023-02-13 04:54:38,618 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, ASSIGN 2023-02-13 04:54:38,619 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=false 2023-02-13 04:54:38,620 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, ASSIGN 2023-02-13 04:54:38,621 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=false 2023-02-13 04:54:38,769 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:54:38,771 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=86ee5aad653c7d36bbe7f423e8fa84f5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:38,771 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264078771"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264078771"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264078771"}]},"ts":"1676264078771"} 2023-02-13 04:54:38,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 86ee5aad653c7d36bbe7f423e8fa84f5, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:38,779 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=e5e5be97dcb4e9a93e2c911fecc4d10b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:38,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264078779"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264078779"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264078779"}]},"ts":"1676264078779"} 2023-02-13 04:54:38,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure e5e5be97dcb4e9a93e2c911fecc4d10b, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:38,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:38,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 86ee5aad653c7d36bbe7f423e8fa84f5, NAME => 'testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 04:54:38,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:38,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:38,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:38,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:38,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:38,933 INFO [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:38,935 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1 2023-02-13 04:54:38,935 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1 2023-02-13 04:54:38,935 INFO [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86ee5aad653c7d36bbe7f423e8fa84f5 columnFamilyName cf1 2023-02-13 04:54:38,970 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a-bottom 2023-02-13 04:54:38,979 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d-bottom 2023-02-13 04:54:38,986 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016-bottom 2023-02-13 04:54:38,993 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0-bottom 2023-02-13 04:54:38,994 INFO [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(310): Store=86ee5aad653c7d36bbe7f423e8fa84f5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:38,994 INFO [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:38,995 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2 2023-02-13 04:54:38,995 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2 2023-02-13 04:54:38,996 INFO [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 86ee5aad653c7d36bbe7f423e8fa84f5 columnFamilyName cf2 2023-02-13 04:54:39,013 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62-bottom 2023-02-13 04:54:39,020 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028-bottom 2023-02-13 04:54:39,027 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761-bottom 2023-02-13 04:54:39,033 DEBUG [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101-bottom 2023-02-13 04:54:39,033 INFO [StoreOpener-86ee5aad653c7d36bbe7f423e8fa84f5-1] regionserver.HStore(310): Store=86ee5aad653c7d36bbe7f423e8fa84f5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:39,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:39,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 86ee5aad653c7d36bbe7f423e8fa84f5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61412795, jitterRate=-0.08487804234027863}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:39,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 86ee5aad653c7d36bbe7f423e8fa84f5: Running coprocessor pre-open hook at 1676264078932Writing region info on filesystem at 1676264078932Initializing all the Stores at 1676264078933 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264078933Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264078933Cleaning up temporary data from old regions at 1676264079037 (+104 ms)Cleaning up detritus from prior splits at 1676264079037Running coprocessor post-open hooks at 1676264079040 (+3 ms)Region opened successfully at 1676264079040 2023-02-13 04:54:39,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5., pid=17, masterSystemTime=1676264078927 2023-02-13 04:54:39,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:39,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:39,044 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 04:54:39,044 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 04:54:39,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,047 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,047 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,047 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1912): 86ee5aad653c7d36bbe7f423e8fa84f5/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:39,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,048 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1912): 86ee5aad653c7d36bbe7f423e8fa84f5/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:39,048 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 86ee5aad653c7d36bbe7f423e8fa84f5/cf1 in testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e5e5be97dcb4e9a93e2c911fecc4d10b, NAME => 'testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 04:54:39,048 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 86ee5aad653c7d36bbe7f423e8fa84f5/cf2 in testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,048 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=86ee5aad653c7d36bbe7f423e8fa84f5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:39,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:39,048 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264079048"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264079048"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264079048"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264079048"}]},"ts":"1676264079048"} 2023-02-13 04:54:39,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,048 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/.tmp, totalSize=19.9 K 2023-02-13 04:54:39,049 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/.tmp, totalSize=19.9 K 2023-02-13 04:54:39,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:39,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,051 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264076374 2023-02-13 04:54:39,051 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264076374 2023-02-13 04:54:39,053 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264076751 2023-02-13 04:54:39,053 INFO [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,053 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264076751 2023-02-13 04:54:39,053 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264076998 2023-02-13 04:54:39,054 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264076998 2023-02-13 04:54:39,055 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264077249 2023-02-13 04:54:39,055 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1 2023-02-13 04:54:39,055 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1 2023-02-13 04:54:39,055 INFO [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5e5be97dcb4e9a93e2c911fecc4d10b columnFamilyName cf1 2023-02-13 04:54:39,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 04:54:39,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 86ee5aad653c7d36bbe7f423e8fa84f5, server=jenkins-hbase4.apache.org,46601,1676264072787 in 278 msec 2023-02-13 04:54:39,056 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264077249 2023-02-13 04:54:39,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, ASSIGN in 439 msec 2023-02-13 04:54:39,074 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] throttle.PressureAwareThroughputController(145): 86ee5aad653c7d36bbe7f423e8fa84f5#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:39,077 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a-top 2023-02-13 04:54:39,077 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] throttle.PressureAwareThroughputController(145): 86ee5aad653c7d36bbe7f423e8fa84f5#cf2#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:39,090 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d-top 2023-02-13 04:54:39,100 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016-top 2023-02-13 04:54:39,100 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/.tmp/cf1/ebf835e1e556499ea23d30d4b19299b1 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/ebf835e1e556499ea23d30d4b19299b1 2023-02-13 04:54:39,105 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0-top 2023-02-13 04:54:39,117 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b 2023-02-13 04:54:39,127 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7 2023-02-13 04:54:39,128 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/.tmp/cf2/451067ccb7d344bda9b8952eb62248ff as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/451067ccb7d344bda9b8952eb62248ff 2023-02-13 04:54:39,132 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 86ee5aad653c7d36bbe7f423e8fa84f5/cf1 of 86ee5aad653c7d36bbe7f423e8fa84f5 into ebf835e1e556499ea23d30d4b19299b1(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:39,132 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 86ee5aad653c7d36bbe7f423e8fa84f5: 2023-02-13 04:54:39,132 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5., storeName=86ee5aad653c7d36bbe7f423e8fa84f5/cf1, priority=12, startTime=1676264079043; duration=0sec 2023-02-13 04:54:39,133 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:39,136 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f 2023-02-13 04:54:39,137 INFO [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(310): Store=e5e5be97dcb4e9a93e2c911fecc4d10b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:39,138 INFO [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,139 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2 2023-02-13 04:54:39,139 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2 2023-02-13 04:54:39,139 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 86ee5aad653c7d36bbe7f423e8fa84f5/cf2 of 86ee5aad653c7d36bbe7f423e8fa84f5 into 451067ccb7d344bda9b8952eb62248ff(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:39,139 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 86ee5aad653c7d36bbe7f423e8fa84f5: 2023-02-13 04:54:39,139 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5., storeName=86ee5aad653c7d36bbe7f423e8fa84f5/cf2, priority=12, startTime=1676264079044; duration=0sec 2023-02-13 04:54:39,139 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:39,139 INFO [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5e5be97dcb4e9a93e2c911fecc4d10b columnFamilyName cf2 2023-02-13 04:54:39,154 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62-top 2023-02-13 04:54:39,159 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028-top 2023-02-13 04:54:39,165 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761-top 2023-02-13 04:54:39,169 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101-top 2023-02-13 04:54:39,174 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3 2023-02-13 04:54:39,179 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3 2023-02-13 04:54:39,184 DEBUG [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4 2023-02-13 04:54:39,184 INFO [StoreOpener-e5e5be97dcb4e9a93e2c911fecc4d10b-1] regionserver.HStore(310): Store=e5e5be97dcb4e9a93e2c911fecc4d10b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:39,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:39,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e5e5be97dcb4e9a93e2c911fecc4d10b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63506750, jitterRate=-0.053675681352615356}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:39,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e5e5be97dcb4e9a93e2c911fecc4d10b: Running coprocessor pre-open hook at 1676264079050Writing region info on filesystem at 1676264079050Initializing all the Stores at 1676264079052 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264079052Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264079053 (+1 ms)Cleaning up temporary data from old regions at 1676264079188 (+135 ms)Cleaning up detritus from prior splits at 1676264079189 (+1 ms)Running coprocessor post-open hooks at 1676264079192 (+3 ms)Region opened successfully at 1676264079192 2023-02-13 04:54:39,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b., pid=18, masterSystemTime=1676264078927 2023-02-13 04:54:39,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:39,194 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:39,195 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:39,195 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:39,195 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1912): e5e5be97dcb4e9a93e2c911fecc4d10b/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:39,195 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,196 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1912): e5e5be97dcb4e9a93e2c911fecc4d10b/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:39,196 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e5e5be97dcb4e9a93e2c911fecc4d10b/cf1 in testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,196 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e5e5be97dcb4e9a93e2c911fecc4d10b/cf2 in testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,196 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/.tmp, totalSize=34.9 K 2023-02-13 04:54:39,196 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/.tmp, totalSize=34.9 K 2023-02-13 04:54:39,197 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264076374 2023-02-13 04:54:39,198 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264076374 2023-02-13 04:54:39,203 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264076751 2023-02-13 04:54:39,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,204 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264076998 2023-02-13 04:54:39,203 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264076751 2023-02-13 04:54:39,204 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=e5e5be97dcb4e9a93e2c911fecc4d10b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:39,204 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264079204"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264079204"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264079204"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264079204"}]},"ts":"1676264079204"} 2023-02-13 04:54:39,204 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264077249 2023-02-13 04:54:39,205 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264076998 2023-02-13 04:54:39,205 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264077488 2023-02-13 04:54:39,205 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264077249 2023-02-13 04:54:39,206 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264077488 2023-02-13 04:54:39,206 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264077723 2023-02-13 04:54:39,207 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264077723 2023-02-13 04:54:39,207 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264077953 2023-02-13 04:54:39,207 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264077953 2023-02-13 04:54:39,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-13 04:54:39,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure e5e5be97dcb4e9a93e2c911fecc4d10b, server=jenkins-hbase4.apache.org,46601,1676264072787 in 426 msec 2023-02-13 04:54:39,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-13 04:54:39,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, ASSIGN in 595 msec 2023-02-13 04:54:39,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b in 1.2410 sec 2023-02-13 04:54:39,235 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] throttle.PressureAwareThroughputController(145): e5e5be97dcb4e9a93e2c911fecc4d10b#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:39,241 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] throttle.PressureAwareThroughputController(145): e5e5be97dcb4e9a93e2c911fecc4d10b#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:39,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46601] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:58126 deadline: 1676264139240, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4. is not online on jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:39,268 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/.tmp/cf2/2ebb1609b1fe44c18fa110c56eab9613 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/2ebb1609b1fe44c18fa110c56eab9613 2023-02-13 04:54:39,270 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/.tmp/cf1/a67bfc4489284b109cb4804f5703d747 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/a67bfc4489284b109cb4804f5703d747 2023-02-13 04:54:39,276 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e5e5be97dcb4e9a93e2c911fecc4d10b/cf2 of e5e5be97dcb4e9a93e2c911fecc4d10b into 2ebb1609b1fe44c18fa110c56eab9613(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:39,276 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e5e5be97dcb4e9a93e2c911fecc4d10b/cf1 of e5e5be97dcb4e9a93e2c911fecc4d10b into a67bfc4489284b109cb4804f5703d747(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:39,277 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e5e5be97dcb4e9a93e2c911fecc4d10b: 2023-02-13 04:54:39,277 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e5e5be97dcb4e9a93e2c911fecc4d10b: 2023-02-13 04:54:39,277 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b., storeName=e5e5be97dcb4e9a93e2c911fecc4d10b/cf2, priority=9, startTime=1676264079194; duration=0sec 2023-02-13 04:54:39,277 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b., storeName=e5e5be97dcb4e9a93e2c911fecc4d10b/cf1, priority=9, startTime=1676264079193; duration=0sec 2023-02-13 04:54:39,277 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:39,277 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:39,361 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 04:54:39,373 INFO [Listener at localhost/33551] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 04:54:39,376 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 04:54:39,408 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:39,419 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,425 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,437 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264079437"}]},"ts":"1676264079437"} 2023-02-13 04:54:39,439 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 04:54:39,442 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 04:54:39,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, UNASSIGN}] 2023-02-13 04:54:39,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 04:54:39,445 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, UNASSIGN 2023-02-13 04:54:39,446 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=e5e5be97dcb4e9a93e2c911fecc4d10b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:39,446 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264079446"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264079446"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264079446"}]},"ts":"1676264079446"} 2023-02-13 04:54:39,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure e5e5be97dcb4e9a93e2c911fecc4d10b, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:39,451 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, UNASSIGN 2023-02-13 04:54:39,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=86ee5aad653c7d36bbe7f423e8fa84f5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:39,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264079452"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264079452"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264079452"}]},"ts":"1676264079452"} 2023-02-13 04:54:39,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 86ee5aad653c7d36bbe7f423e8fa84f5, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:39,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 04:54:39,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e5e5be97dcb4e9a93e2c911fecc4d10b, disabling compactions & flushes 2023-02-13 04:54:39,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. after waiting 0 ms 2023-02-13 04:54:39,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,614 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7] to archive 2023-02-13 04:54:39,615 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:39,620 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,622 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,624 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,625 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,627 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-150a0082b404460795717fa538d7d88b 2023-02-13 04:54:39,629 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ed6e91d74655464b91bf13df070efd7f 2023-02-13 04:54:39,631 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-5b3e423638f8481fad9715a1a35a99f7 2023-02-13 04:54:39,658 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4] to archive 2023-02-13 04:54:39,659 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:39,662 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,663 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,665 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,667 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,668 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-97a7c59c8b8b47d2a8d4b711d799a8a3 2023-02-13 04:54:39,670 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-ec837eb3ceee4b6ca8f6cac47af992d3 2023-02-13 04:54:39,671 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/testSplitTableRegionUnevenDaughter=3480cf0bddc2adc7fa210c7f8e2c70b4-f37a20e72fcb4641b1b6aeb7f5ff4aa4 2023-02-13 04:54:39,676 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:54:39,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:39,677 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b. 2023-02-13 04:54:39,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e5e5be97dcb4e9a93e2c911fecc4d10b: Waiting for close lock at 1676264079601Running coprocessor pre-close hooks at 1676264079601Disabling compacts and flushes for region at 1676264079601Disabling writes for close at 1676264079601Writing region close event to WAL at 1676264079672 (+71 ms)Running coprocessor post-close hooks at 1676264079677 (+5 ms)Closed at 1676264079677 2023-02-13 04:54:39,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 86ee5aad653c7d36bbe7f423e8fa84f5, disabling compactions & flushes 2023-02-13 04:54:39,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. after waiting 0 ms 2023-02-13 04:54:39,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=e5e5be97dcb4e9a93e2c911fecc4d10b, regionState=CLOSED 2023-02-13 04:54:39,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264079681"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264079681"}]},"ts":"1676264079681"} 2023-02-13 04:54:39,681 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0-bottom] to archive 2023-02-13 04:54:39,682 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:39,686 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/43f598b647bf45209897fb6ac4891a3d.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 04:54:39,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure e5e5be97dcb4e9a93e2c911fecc4d10b, server=jenkins-hbase4.apache.org,46601,1676264072787 in 236 msec 2023-02-13 04:54:39,689 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/51c8f9134b0148189f7596cd45ded016.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e5e5be97dcb4e9a93e2c911fecc4d10b, UNASSIGN in 243 msec 2023-02-13 04:54:39,691 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/152faa931d504454bc3df0dcf55a668a.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,693 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/c2977b24e41a4e4983bc3540b82d32f0.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,695 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761-bottom] to archive 2023-02-13 04:54:39,696 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:39,698 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/58376666f832426e96c726ba640dc028.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,699 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/d6daeb1e1e9f4821b538f838c5727101.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,701 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,702 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/6a365f11c4c44fe4a7efb5ea32d5d761.3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:54:39,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:39,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5. 2023-02-13 04:54:39,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 86ee5aad653c7d36bbe7f423e8fa84f5: Waiting for close lock at 1676264079680Running coprocessor pre-close hooks at 1676264079680Disabling compacts and flushes for region at 1676264079680Disabling writes for close at 1676264079680Writing region close event to WAL at 1676264079704 (+24 ms)Running coprocessor post-close hooks at 1676264079708 (+4 ms)Closed at 1676264079708 2023-02-13 04:54:39,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=86ee5aad653c7d36bbe7f423e8fa84f5, regionState=CLOSED 2023-02-13 04:54:39,712 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676264079712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264079712"}]},"ts":"1676264079712"} 2023-02-13 04:54:39,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 04:54:39,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 86ee5aad653c7d36bbe7f423e8fa84f5, server=jenkins-hbase4.apache.org,46601,1676264072787 in 259 msec 2023-02-13 04:54:39,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 04:54:39,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=86ee5aad653c7d36bbe7f423e8fa84f5, UNASSIGN in 274 msec 2023-02-13 04:54:39,722 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264079722"}]},"ts":"1676264079722"} 2023-02-13 04:54:39,724 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 04:54:39,725 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 04:54:39,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 299 msec 2023-02-13 04:54:39,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 04:54:39,751 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 04:54:39,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,768 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,771 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 04:54:39,780 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,781 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,784 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,788 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/recovered.edits] 2023-02-13 04:54:39,789 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/recovered.edits] 2023-02-13 04:54:39,791 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/recovered.edits] 2023-02-13 04:54:39,797 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/ebf835e1e556499ea23d30d4b19299b1 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf1/ebf835e1e556499ea23d30d4b19299b1 2023-02-13 04:54:39,797 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/150a0082b404460795717fa538d7d88b 2023-02-13 04:54:39,799 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/a67bfc4489284b109cb4804f5703d747 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf1/a67bfc4489284b109cb4804f5703d747 2023-02-13 04:54:39,800 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/152faa931d504454bc3df0dcf55a668a 2023-02-13 04:54:39,802 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/451067ccb7d344bda9b8952eb62248ff to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/cf2/451067ccb7d344bda9b8952eb62248ff 2023-02-13 04:54:39,802 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/43f598b647bf45209897fb6ac4891a3d 2023-02-13 04:54:39,803 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/2ebb1609b1fe44c18fa110c56eab9613 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/cf2/2ebb1609b1fe44c18fa110c56eab9613 2023-02-13 04:54:39,804 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/51c8f9134b0148189f7596cd45ded016 2023-02-13 04:54:39,807 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/5b3e423638f8481fad9715a1a35a99f7 2023-02-13 04:54:39,807 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5/recovered.edits/90.seqid 2023-02-13 04:54:39,809 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/86ee5aad653c7d36bbe7f423e8fa84f5 2023-02-13 04:54:39,809 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b/recovered.edits/90.seqid 2023-02-13 04:54:39,809 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/c2977b24e41a4e4983bc3540b82d32f0 2023-02-13 04:54:39,810 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:39,811 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf1/ed6e91d74655464b91bf13df070efd7f 2023-02-13 04:54:39,815 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/0cb81b345e4a408dbbdf7a75b2bc4e62 2023-02-13 04:54:39,817 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/58376666f832426e96c726ba640dc028 2023-02-13 04:54:39,819 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/6a365f11c4c44fe4a7efb5ea32d5d761 2023-02-13 04:54:39,821 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/97a7c59c8b8b47d2a8d4b711d799a8a3 2023-02-13 04:54:39,823 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/d6daeb1e1e9f4821b538f838c5727101 2023-02-13 04:54:39,824 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/ec837eb3ceee4b6ca8f6cac47af992d3 2023-02-13 04:54:39,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/cf2/f37a20e72fcb4641b1b6aeb7f5ff4aa4 2023-02-13 04:54:39,829 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/recovered.edits/85.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4/recovered.edits/85.seqid 2023-02-13 04:54:39,830 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionUnevenDaughter/3480cf0bddc2adc7fa210c7f8e2c70b4 2023-02-13 04:54:39,830 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 04:54:39,833 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,856 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 04:54:39,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 04:54:39,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 04:54:39,902 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,902 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 04:54:39,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264079902"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:39,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264079902"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:39,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264079902"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:39,906 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:54:39,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3480cf0bddc2adc7fa210c7f8e2c70b4, NAME => 'testSplitTableRegionUnevenDaughter,,1676264075607.3480cf0bddc2adc7fa210c7f8e2c70b4.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 86ee5aad653c7d36bbe7f423e8fa84f5, NAME => 'testSplitTableRegionUnevenDaughter,,1676264077974.86ee5aad653c7d36bbe7f423e8fa84f5.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => e5e5be97dcb4e9a93e2c911fecc4d10b, NAME => 'testSplitTableRegionUnevenDaughter,26,1676264077974.e5e5be97dcb4e9a93e2c911fecc4d10b.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 04:54:39,906 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 04:54:39,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264079906"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:39,908 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 04:54:39,912 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:39,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 153 msec 2023-02-13 04:54:40,009 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:54:40,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 04:54:40,079 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 04:54:40,099 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 357) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52456 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52506 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fc298c7-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:53616 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:53638 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52524 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:53590 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52758 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:34920 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52482 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52770 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:46601-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:34618 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fc298c7-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:53414 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:34584 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:34650 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:52570 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1869953903_15 at /127.0.0.1:36266 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 264), ProcessCount=168 (was 168), AvailableMemoryMB=3886 (was 4038) 2023-02-13 04:54:40,114 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=168, AvailableMemoryMB=3886 2023-02-13 04:54:40,114 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:40,119 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 04:54:40,120 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 04:54:40,120 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:54:40,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:54:40,122 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:40,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:54:40,125 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:54:40,128 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,129 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 empty. 2023-02-13 04:54:40,129 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,129 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 04:54:40,545 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:40,547 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8ad8f350a2b6f56d4128dc02f389ea94, NAME => 'testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:54:40,558 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:40,558 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 8ad8f350a2b6f56d4128dc02f389ea94, disabling compactions & flushes 2023-02-13 04:54:40,558 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,558 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,558 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. after waiting 0 ms 2023-02-13 04:54:40,558 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,558 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,558 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 8ad8f350a2b6f56d4128dc02f389ea94: Waiting for close lock at 1676264080558Disabling compacts and flushes for region at 1676264080558Disabling writes for close at 1676264080558Writing region close event to WAL at 1676264080558Closed at 1676264080558 2023-02-13 04:54:40,562 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:54:40,563 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264080563"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264080563"}]},"ts":"1676264080563"} 2023-02-13 04:54:40,565 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:54:40,566 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:54:40,566 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264080566"}]},"ts":"1676264080566"} 2023-02-13 04:54:40,567 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 04:54:40,572 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:40,573 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:40,573 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:40,573 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:40,573 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:40,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, ASSIGN}] 2023-02-13 04:54:40,575 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, ASSIGN 2023-02-13 04:54:40,576 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:54:40,726 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:40,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=8ad8f350a2b6f56d4128dc02f389ea94, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:40,728 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264080728"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264080728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264080728"}]},"ts":"1676264080728"} 2023-02-13 04:54:40,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:40,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8ad8f350a2b6f56d4128dc02f389ea94, NAME => 'testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:40,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:40,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:40,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,890 INFO [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,893 DEBUG [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1 2023-02-13 04:54:40,893 DEBUG [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1 2023-02-13 04:54:40,894 INFO [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ad8f350a2b6f56d4128dc02f389ea94 columnFamilyName cf1 2023-02-13 04:54:40,896 INFO [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] regionserver.HStore(310): Store=8ad8f350a2b6f56d4128dc02f389ea94/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:40,897 INFO [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,898 DEBUG [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2 2023-02-13 04:54:40,898 DEBUG [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2 2023-02-13 04:54:40,899 INFO [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ad8f350a2b6f56d4128dc02f389ea94 columnFamilyName cf2 2023-02-13 04:54:40,900 INFO [StoreOpener-8ad8f350a2b6f56d4128dc02f389ea94-1] regionserver.HStore(310): Store=8ad8f350a2b6f56d4128dc02f389ea94/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:40,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:40,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:40,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:40,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8ad8f350a2b6f56d4128dc02f389ea94; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75240459, jitterRate=0.12117020785808563}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:40,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8ad8f350a2b6f56d4128dc02f389ea94: Running coprocessor pre-open hook at 1676264080889Writing region info on filesystem at 1676264080889Initializing all the Stores at 1676264080890 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264080890Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264080890Cleaning up temporary data from old regions at 1676264080902 (+12 ms)Cleaning up detritus from prior splits at 1676264080903 (+1 ms)Running coprocessor post-open hooks at 1676264080908 (+5 ms)Region opened successfully at 1676264080908 2023-02-13 04:54:40,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94., pid=27, masterSystemTime=1676264080883 2023-02-13 04:54:40,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:40,912 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=8ad8f350a2b6f56d4128dc02f389ea94, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:40,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264080911"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264080911"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264080911"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264080911"}]},"ts":"1676264080911"} 2023-02-13 04:54:40,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 04:54:40,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 in 183 msec 2023-02-13 04:54:40,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 04:54:40,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, ASSIGN in 343 msec 2023-02-13 04:54:40,920 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:54:40,920 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264080920"}]},"ts":"1676264080920"} 2023-02-13 04:54:40,922 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 04:54:40,924 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:54:40,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 803 msec 2023-02-13 04:54:41,135 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:41,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 04:54:41,138 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 04:54:41,138 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,139 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 04:54:41,139 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 04:54:41,139 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,139 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,140 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,140 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,140 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,140 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,141 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,141 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,142 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,143 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,143 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,143 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,143 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,144 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,144 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,144 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,144 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,144 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,144 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,145 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,145 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:41,145 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:41,145 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,145 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. started... 2023-02-13 04:54:41,146 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:54:41,146 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,147 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,147 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,148 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,148 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,148 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,158 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/3152b757c29544aa9b6496d4fbda6445 2023-02-13 04:54:41,174 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/9bfa185affaa4d878c1cacd81770c433 2023-02-13 04:54:41,179 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/3152b757c29544aa9b6496d4fbda6445 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 2023-02-13 04:54:41,184 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:41,184 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/9bfa185affaa4d878c1cacd81770c433 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 2023-02-13 04:54:41,189 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:41,190 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 44ms, sequenceid=6, compaction requested=false 2023-02-13 04:54:41,190 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 04:54:41,191 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ad8f350a2b6f56d4128dc02f389ea94: 2023-02-13 04:54:41,191 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,191 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:41,191 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:41,191 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,191 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,191 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,193 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,193 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 04:54:41,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27986485[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:41,195 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 04:54:41,195 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,196 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,196 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,196 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,197 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,197 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:41,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,197 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:41,199 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,199 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,199 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,200 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,200 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,200 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,200 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,200 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,200 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,201 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:54:41,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 04:54:41,205 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 04:54:41,205 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:41,205 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:41,205 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 04:54:41,205 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,207 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,207 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,208 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,208 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,210 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,211 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,211 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,214 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,215 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 04:54:41,215 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:41,214 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,216 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,216 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,216 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:41,216 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:41,316 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:41,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:41,336 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:41,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:41,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 04:54:41,338 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 04:54:41,338 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,339 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 04:54:41,339 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 04:54:41,340 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,340 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,341 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,341 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,341 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,341 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,341 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,342 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:41,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,343 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,343 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,344 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,344 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,344 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,345 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,345 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,345 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,345 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,346 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,345 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:41,346 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:41,345 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,346 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,346 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. started... 2023-02-13 04:54:41,346 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,346 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,346 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:41,349 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,350 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,350 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,350 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,350 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,350 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,359 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/d88fb96b4e0547b9adf5b5d1db626623 2023-02-13 04:54:41,373 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/9b28c5125c95434e931019ddf2923ee7 2023-02-13 04:54:41,378 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/d88fb96b4e0547b9adf5b5d1db626623 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 2023-02-13 04:54:41,383 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:41,384 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/9b28c5125c95434e931019ddf2923ee7 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 2023-02-13 04:54:41,388 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:41,389 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 43ms, sequenceid=19, compaction requested=false 2023-02-13 04:54:41,390 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ad8f350a2b6f56d4128dc02f389ea94: 2023-02-13 04:54:41,390 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,390 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:41,390 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:41,390 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,390 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,390 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,391 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,392 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,392 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 04:54:41,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e7f620a[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:41,394 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 04:54:41,394 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,395 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,395 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,396 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,395 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:41,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,396 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:41,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:41,396 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:41,398 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,398 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,398 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,398 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,406 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,406 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,406 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,407 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,407 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,407 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:54:41,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 04:54:41,411 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 04:54:41,411 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:41,411 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:41,411 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 04:54:41,411 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,414 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,415 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,415 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,416 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,418 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,418 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,418 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 04:54:41,423 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:41,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,423 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:41,423 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:41,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,421 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,523 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:41,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:41,545 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:41,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:41,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 04:54:41,547 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 04:54:41,547 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,548 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 04:54:41,548 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 04:54:41,548 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,549 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,550 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,550 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,550 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,550 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,551 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,552 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,552 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,552 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,552 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,554 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:41,554 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. started... 2023-02-13 04:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,554 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:41,555 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,557 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,557 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,557 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,557 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,557 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,567 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/981c8d1730324863bf7f6e8d267d100b 2023-02-13 04:54:41,582 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/1b7c7c77807b47debc75fd9183ecabd2 2023-02-13 04:54:41,588 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/981c8d1730324863bf7f6e8d267d100b as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b 2023-02-13 04:54:41,594 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:41,595 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/1b7c7c77807b47debc75fd9183ecabd2 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 2023-02-13 04:54:41,600 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:41,601 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 47ms, sequenceid=32, compaction requested=true 2023-02-13 04:54:41,601 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ad8f350a2b6f56d4128dc02f389ea94: 2023-02-13 04:54:41,601 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:41,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:41,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,601 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,603 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,603 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,604 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 04:54:41,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45371be8[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:41,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 04:54:41,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:41,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,609 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:41,608 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:41,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,612 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,612 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,612 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,612 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,612 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,612 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,613 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:54:41,617 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 04:54:41,617 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 04:54:41,617 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:41,617 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:41,617 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 04:54:41,617 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,619 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,620 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,620 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,621 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,624 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,624 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,625 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,629 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,629 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 04:54:41,630 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:41,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,629 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:41,631 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:41,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,731 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:41,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:41,752 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:41,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:41,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 04:54:41,754 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 04:54:41,754 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,755 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 04:54:41,755 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 04:54:41,755 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,755 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,757 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,757 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,757 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,757 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,757 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,757 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,758 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,759 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,761 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,761 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:41,764 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:41,764 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,764 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. started... 2023-02-13 04:54:41,764 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:41,772 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,773 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,773 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,774 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,774 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,774 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,776 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/55d71b080bf44c4a951c1fd701770f22 2023-02-13 04:54:41,789 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/08fd259382374a54a96419d51b051816 2023-02-13 04:54:41,794 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/55d71b080bf44c4a951c1fd701770f22 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 2023-02-13 04:54:41,799 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:41,800 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/08fd259382374a54a96419d51b051816 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 2023-02-13 04:54:41,804 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:41,805 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 41ms, sequenceid=45, compaction requested=true 2023-02-13 04:54:41,806 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ad8f350a2b6f56d4128dc02f389ea94: 2023-02-13 04:54:41,806 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:41,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:41,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,807 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,807 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,808 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,808 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 04:54:41,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@786c0b4f[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:41,810 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 04:54:41,810 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,811 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:41,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:41,811 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:41,813 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,813 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,813 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,814 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,814 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:41,814 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,815 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,814 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,815 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,815 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:41,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,820 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 04:54:41,821 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 04:54:41,821 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:41,821 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 04:54:41,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:54:41,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 04:54:41,824 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 04:54:41,824 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:41,824 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:41,824 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 04:54:41,824 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,826 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:41,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:41,826 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:41,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,830 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:41,831 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:41,833 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:41,833 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,834 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,837 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,837 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,837 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:41,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 04:54:41,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:41,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,838 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:41,838 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:41,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,839 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:41,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:41,939 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:41,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:41,960 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:41,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:41,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 04:54:41,962 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 04:54:41,962 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,962 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 04:54:41,962 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 04:54:41,963 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,963 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,965 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,965 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,965 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:41,965 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:41,966 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:41,966 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:41,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,967 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:41,968 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:41,968 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:41,968 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:41,968 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. started... 2023-02-13 04:54:41,968 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:41,969 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:41,970 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:41,979 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/587a1600f61e487eb6eef39085e1250f 2023-02-13 04:54:41,993 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/ada18712032345cda6af2c7e9af5683a 2023-02-13 04:54:41,998 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/587a1600f61e487eb6eef39085e1250f as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f 2023-02-13 04:54:42,003 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:42,004 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/ada18712032345cda6af2c7e9af5683a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a 2023-02-13 04:54:42,010 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:42,011 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 43ms, sequenceid=58, compaction requested=true 2023-02-13 04:54:42,011 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ad8f350a2b6f56d4128dc02f389ea94: 2023-02-13 04:54:42,011 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:42,011 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:42,011 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:42,011 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,011 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:42,011 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:42,013 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,013 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:42,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:42,013 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,013 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:42,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:42,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:42,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:42,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 04:54:42,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1cb4f48b[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:42,015 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 04:54:42,015 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:42,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:42,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:42,018 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:42,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:42,018 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:42,019 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:42,019 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:42,020 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:42,020 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,020 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:42,021 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,021 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:42,021 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:42,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:42,021 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:42,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:42,021 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:42,021 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,021 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:42,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:42,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:42,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:54:42,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 04:54:42,024 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 04:54:42,024 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:42,024 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:42,024 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 04:54:42,024 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:42,027 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,027 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,027 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:42,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:42,027 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,027 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,027 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,028 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,028 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:42,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,028 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,029 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:42,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,036 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,036 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,036 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 04:54:42,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,037 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:42,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:42,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,037 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:42,038 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,038 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,038 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,138 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:42,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:42,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:42,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:42,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 04:54:42,161 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 04:54:42,161 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:42,161 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 04:54:42,161 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 04:54:42,162 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,162 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,163 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,163 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,163 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,163 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,164 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,164 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:42,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 04:54:42,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:42,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:42,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:42,166 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 04:54:42,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:42,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 04:54:42,166 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:42,167 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:42,167 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:42,168 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:42,168 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. started... 2023-02-13 04:54:42,168 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:42,169 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:42,180 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/1bd160361bfb4bffa7d04988f01beb7d 2023-02-13 04:54:42,196 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b 2023-02-13 04:54:42,202 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/1bd160361bfb4bffa7d04988f01beb7d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d 2023-02-13 04:54:42,207 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:42,208 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b 2023-02-13 04:54:42,212 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:42,214 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 45ms, sequenceid=71, compaction requested=true 2023-02-13 04:54:42,214 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8ad8f350a2b6f56d4128dc02f389ea94: 2023-02-13 04:54:42,214 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:42,214 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:42,214 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:42,214 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,214 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 04:54:42,214 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 04:54:42,216 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,216 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:42,216 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,216 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:42,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 04:54:42,218 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29eeaa25[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:42,218 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 04:54:42,218 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,219 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:42,219 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 04:54:42,220 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:42,222 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:42,222 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,222 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:42,223 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:42,223 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,223 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:42,223 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 04:54:42,223 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,223 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:42,223 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 04:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:54:42,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 04:54:42,226 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 04:54:42,227 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:42,227 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:42,227 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 04:54:42,227 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:42,228 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:42,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:42,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,228 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:42,229 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,229 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,229 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:42,230 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,231 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:42,232 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 04:54:42,232 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,235 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:42,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 04:54:42,235 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,235 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:42,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,235 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:42,236 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:42,236 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:42,236 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:42,236 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 04:54:42,236 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 04:54:42,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:42,336 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:42,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:42,600 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 04:54:42,601 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:42,603 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:42,605 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:42,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:42,853 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,854 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:42,854 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:42,855 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:42,857 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 04:54:42,857 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:42,857 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:42,858 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:42,858 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:42,858 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:42,861 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 04:54:42,863 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:42,863 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:42,863 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:42,864 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:42,864 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:42,864 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:42,865 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:42,865 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:42,866 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:42,866 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,866 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:42,866 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,866 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,866 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,867 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 04:54:42,867 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:42,868 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:42,869 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264112869 2023-02-13 04:54:42,870 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:42,870 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,874 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:42,874 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:42,874 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:42,874 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264142874 2023-02-13 04:54:42,874 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264202874 2023-02-13 04:54:42,874 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 04:54:42,875 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:42,877 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:42,877 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:42,877 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:42,877 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:43,125 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:43,125 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:43,126 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:43,126 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:43,128 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 04:54:43,128 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:43,129 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:43,129 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:43,129 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:43,130 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,130 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:43,130 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,130 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,131 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:43,131 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,132 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,132 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,132 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,132 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,132 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:43,133 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:43,133 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:43,138 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264113138 2023-02-13 04:54:43,138 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:43,139 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,142 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:43,143 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:43,143 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:43,143 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264143143 2023-02-13 04:54:43,143 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264203143 2023-02-13 04:54:43,143 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 04:54:43,144 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:43,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN}] 2023-02-13 04:54:43,146 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:43,146 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:43,146 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:43,394 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:43,394 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:43,395 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:43,397 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 04:54:43,397 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:43,398 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:43,398 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:43,398 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:43,399 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,400 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:43,400 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,400 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,400 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:43,401 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,401 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,401 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,401 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,401 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,401 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:43,402 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:43,402 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:43,402 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264113402 2023-02-13 04:54:43,403 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:43,404 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,406 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:43,407 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:43,407 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:43,407 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264143407 2023-02-13 04:54:43,407 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264203407 2023-02-13 04:54:43,407 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 04:54:43,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN}] 2023-02-13 04:54:43,407 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:43,409 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,410 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:43,410 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264083410"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264083410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264083410"}]},"ts":"1676264083410"} 2023-02-13 04:54:43,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:43,412 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:43,412 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:43,412 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:43,657 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,657 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:43,657 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:43,657 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:43,659 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:43,659 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:43,659 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:43,659 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:43,659 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:43,660 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:43,662 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:43,662 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 04:54:43,662 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:43,663 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:43,663 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:43,664 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:43,664 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,665 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:43,665 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,665 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,666 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:43,667 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,667 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,668 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,668 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,668 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,668 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,668 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,668 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,668 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 04:54:43,669 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:43,670 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:43,670 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94 to restore RIT 2023-02-13 04:54:43,670 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264113670 2023-02-13 04:54:43,671 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:43,671 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,674 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:43,674 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:43,674 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:43,674 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264143674 2023-02-13 04:54:43,674 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264203674 2023-02-13 04:54:43,674 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:43,675 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:43,675 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264083674"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264083674"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264083674"}]},"ts":"1676264083674"} 2023-02-13 04:54:43,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:43,676 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:43,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:43,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8ad8f350a2b6f56d4128dc02f389ea94, disabling compactions & flushes 2023-02-13 04:54:43,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:43,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:43,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. after waiting 0 ms 2023-02-13 04:54:43,830 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:43,830 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 8ad8f350a2b6f56d4128dc02f389ea94 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:54:43,843 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/53291da7d9a242ee8bfb1e7ee6453b91 2023-02-13 04:54:43,862 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/3bfa7c4b8e84456580b6fd5621dd1dde 2023-02-13 04:54:43,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf1/53291da7d9a242ee8bfb1e7ee6453b91 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 2023-02-13 04:54:43,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:54:43,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/.tmp/cf2/3bfa7c4b8e84456580b6fd5621dd1dde as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde 2023-02-13 04:54:43,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:54:43,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 49ms, sequenceid=82, compaction requested=true 2023-02-13 04:54:43,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:54:43,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:43,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. 2023-02-13 04:54:43,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8ad8f350a2b6f56d4128dc02f389ea94: Waiting for close lock at 1676264083830Running coprocessor pre-close hooks at 1676264083830Disabling compacts and flushes for region at 1676264083830Disabling writes for close at 1676264083830Obtaining lock to block concurrent updates at 1676264083830Preparing flush snapshotting stores in 8ad8f350a2b6f56d4128dc02f389ea94 at 1676264083830Finished memstore snapshotting testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676264083831 (+1 ms)Flushing stores of testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. at 1676264083831Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf1: creating writer at 1676264083832 (+1 ms)Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf1: appending metadata at 1676264083835 (+3 ms)Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf1: closing flushed file at 1676264083835Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf2: creating writer at 1676264083849 (+14 ms)Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf2: appending metadata at 1676264083851 (+2 ms)Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf2: closing flushed file at 1676264083851Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf1: reopening flushed file at 1676264083869 (+18 ms)Flushing 8ad8f350a2b6f56d4128dc02f389ea94/cf2: reopening flushed file at 1676264083874 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 8ad8f350a2b6f56d4128dc02f389ea94 in 49ms, sequenceid=82, compaction requested=true at 1676264083879 (+5 ms)Writing region close event to WAL at 1676264083898 (+19 ms)Running coprocessor post-close hooks at 1676264083902 (+4 ms)Closed at 1676264083902 2023-02-13 04:54:43,905 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:43,906 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED 2023-02-13 04:54:43,906 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264083905"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264083905"}]},"ts":"1676264083905"} 2023-02-13 04:54:43,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:43,907 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:43,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:43,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:43,925 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:43,925 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:43,926 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:43,927 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:43,927 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 04:54:43,927 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 04:54:43,928 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:43,928 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:43,928 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:43,928 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:43,929 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,929 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:43,929 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,929 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:43,930 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:43,930 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,931 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,931 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:43,931 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,931 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,931 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,931 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:43,931 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,931 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,931 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:43,931 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:43,932 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:43,932 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:43,932 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94 to restore RIT 2023-02-13 04:54:43,933 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264113933 2023-02-13 04:54:43,934 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:43,934 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:43,936 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:43,936 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:43,936 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:43,937 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264143937 2023-02-13 04:54:43,937 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264203937 2023-02-13 04:54:43,937 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:43,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:43,937 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264083937"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264083937"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264083937"}]},"ts":"1676264083937"} 2023-02-13 04:54:43,939 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:43,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 04:54:43,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 in 263 msec 2023-02-13 04:54:43,942 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:43,942 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:43,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:43,942 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:44,187 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,187 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:44,187 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:44,187 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:44,187 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:44,187 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:44,188 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:44,188 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:44,188 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:44,188 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:44,190 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:44,190 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 04:54:44,190 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:44,190 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:44,191 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:44,191 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:44,191 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:44,192 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,192 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:44,192 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,192 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,193 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:44,193 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,193 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:44,194 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:44,194 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:44,194 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,194 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,194 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,194 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,194 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,194 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:44,195 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:44,195 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:44,195 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94 to restore RIT 2023-02-13 04:54:44,196 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264114196 2023-02-13 04:54:44,197 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:44,197 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:44,199 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:44,199 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:44,199 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:44,199 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264144199 2023-02-13 04:54:44,199 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264204199 2023-02-13 04:54:44,199 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:54:44,200 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:44,201 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:44,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 04:54:44,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN in 794 msec 2023-02-13 04:54:44,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:44,207 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:44,207 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:44,450 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:44,450 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:44,451 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:44,451 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:44,453 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:44,453 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:44,453 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:44,453 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:44,453 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:44,454 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:44,454 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:44,454 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,454 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:44,455 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,455 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,455 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:44,456 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,456 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:44,456 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:44,456 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,456 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,456 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,456 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:44,457 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:44,457 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:44,458 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264114458 2023-02-13 04:54:44,459 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:44,459 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,462 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:44,462 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:44,462 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:44,462 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264144462 2023-02-13 04:54:44,462 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264204462 2023-02-13 04:54:44,462 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:54:44,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,467 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:44,478 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=8ad8f350a2b6f56d4128dc02f389ea94, threads=14 2023-02-13 04:54:44,479 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,479 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,479 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,479 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,479 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,480 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,480 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,485 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,485 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,485 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,485 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,485 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,486 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,486 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,643 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,655 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,657 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,664 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,675 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,683 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,688 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,695 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,701 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,701 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,702 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,703 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,704 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,707 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,707 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 8ad8f350a2b6f56d4128dc02f389ea94 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:54:44,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:44,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:44,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:44,962 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:44,963 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:44,963 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:44,964 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:44,965 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 04:54:44,965 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:44,965 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:44,966 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:44,966 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:44,967 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:44,967 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:44,967 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,968 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:44,968 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,969 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:44,969 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:44,970 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,970 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:44,970 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:44,970 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:44,970 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,970 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,970 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,970 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,970 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:44,970 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:54:44,971 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:44,972 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:44,972 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264114972 2023-02-13 04:54:44,972 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:44,973 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:44,976 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:44,976 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:44,976 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:44,976 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264144976 2023-02-13 04:54:44,976 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264204976 2023-02-13 04:54:44,976 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 04:54:44,987 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=8ad8f350a2b6f56d4128dc02f389ea94, threads=14 2023-02-13 04:54:44,988 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,988 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,988 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,992 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,992 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,992 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,993 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,993 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,996 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,996 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:44,996 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,004 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,004 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,004 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,139 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,179 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,180 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:54:45,198 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,201 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,288 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,292 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,301 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,301 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,308 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,310 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,312 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,333 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,333 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,694 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 for region: 8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:45,694 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 8ad8f350a2b6f56d4128dc02f389ea94 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:54:45,732 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:45,737 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:54:45,739 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:54:45,739 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:45,739 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:45,739 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:45,977 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,977 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:45,977 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:45,977 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:45,978 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:45,978 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:45,978 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:45,978 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:45,978 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:45,979 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:45,980 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 04:54:45,980 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:45,980 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:45,981 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:45,981 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:45,981 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:45,981 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:45,982 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:45,982 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:45,982 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:45,982 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:45,983 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:45,983 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,983 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:45,983 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:45,984 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:45,984 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,984 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,984 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,984 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:45,984 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:45,984 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:45,985 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:45,985 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:45,986 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264115986 2023-02-13 04:54:45,986 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:45,987 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,990 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:45,990 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:45,990 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:45,990 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264145990 2023-02-13 04:54:45,990 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264205990 2023-02-13 04:54:45,990 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 04:54:45,993 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:45,995 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:45,995 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:45,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:45,995 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:46,240 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:46,241 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:46,241 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:46,241 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:46,242 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:46,242 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:46,242 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:46,242 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:46,242 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:46,243 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:46,244 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 04:54:46,244 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:46,244 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:46,245 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:46,245 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:46,245 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:46,246 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:46,246 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:46,246 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:46,246 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:46,247 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:46,247 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:46,248 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,248 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN 2023-02-13 04:54:46,248 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:46,248 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:46,248 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,248 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,248 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,249 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8ad8f350a2b6f56d4128dc02f389ea94, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:46,249 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 8ad8f350a2b6f56d4128dc02f389ea94, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:46,249 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:46,249 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:46,250 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:46,250 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264116250 2023-02-13 04:54:46,252 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:46,252 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,255 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:46,255 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=8ad8f350a2b6f56d4128dc02f389ea94, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:54:46,255 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:46,255 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264146255 2023-02-13 04:54:46,255 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264206255 2023-02-13 04:54:46,255 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:46,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264086258"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676264086258"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676264086258"}]},"ts":"1676264086258"} 2023-02-13 04:54:46,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264086258"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264086258"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264086258"}]},"ts":"1676264086258"} 2023-02-13 04:54:46,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264086258"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264086258"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264086258"}]},"ts":"1676264086258"} 2023-02-13 04:54:46,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, ASSIGN}] 2023-02-13 04:54:46,268 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, ASSIGN 2023-02-13 04:54:46,270 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:54:46,272 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, ASSIGN 2023-02-13 04:54:46,273 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:54:46,420 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:54:46,421 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=cc7e469ee5eb3889f5d7044238390067, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:46,422 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264086421"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264086421"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264086421"}]},"ts":"1676264086421"} 2023-02-13 04:54:46,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure cc7e469ee5eb3889f5d7044238390067, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:46,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a6deada879885dca6c4b89e4a7394b55, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:46,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264086427"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264086427"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264086427"}]},"ts":"1676264086427"} 2023-02-13 04:54:46,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure a6deada879885dca6c4b89e4a7394b55, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:46,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6deada879885dca6c4b89e4a7394b55, NAME => 'testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 04:54:46,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:46,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:46,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,581 INFO [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,582 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1 2023-02-13 04:54:46,582 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1 2023-02-13 04:54:46,582 INFO [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6deada879885dca6c4b89e4a7394b55 columnFamilyName cf1 2023-02-13 04:54:46,599 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d-top 2023-02-13 04:54:46,604 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445-top 2023-02-13 04:54:46,608 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91-top 2023-02-13 04:54:46,614 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22-top 2023-02-13 04:54:46,620 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f-top 2023-02-13 04:54:46,625 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b-top 2023-02-13 04:54:46,629 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623-top 2023-02-13 04:54:46,629 INFO [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(310): Store=a6deada879885dca6c4b89e4a7394b55/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:46,630 INFO [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,630 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2 2023-02-13 04:54:46,630 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2 2023-02-13 04:54:46,631 INFO [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6deada879885dca6c4b89e4a7394b55 columnFamilyName cf2 2023-02-13 04:54:46,645 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816-top 2023-02-13 04:54:46,649 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2-top 2023-02-13 04:54:46,654 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde-top 2023-02-13 04:54:46,658 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b-top 2023-02-13 04:54:46,662 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7-top 2023-02-13 04:54:46,667 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433-top 2023-02-13 04:54:46,671 DEBUG [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a-top 2023-02-13 04:54:46,671 INFO [StoreOpener-a6deada879885dca6c4b89e4a7394b55-1] regionserver.HStore(310): Store=a6deada879885dca6c4b89e4a7394b55/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:46,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:46,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:46,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a6deada879885dca6c4b89e4a7394b55; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67539294, jitterRate=0.006413906812667847}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:46,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a6deada879885dca6c4b89e4a7394b55: Running coprocessor pre-open hook at 1676264086579Writing region info on filesystem at 1676264086580 (+1 ms)Initializing all the Stores at 1676264086580Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264086580Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264086580Cleaning up temporary data from old regions at 1676264086674 (+94 ms)Cleaning up detritus from prior splits at 1676264086674Running coprocessor post-open hooks at 1676264086676 (+2 ms)Region opened successfully at 1676264086676 2023-02-13 04:54:46,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55., pid=34, masterSystemTime=1676264086575 2023-02-13 04:54:46,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:46,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:46,679 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:46,679 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:46,679 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,679 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): a6deada879885dca6c4b89e4a7394b55/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:46,679 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,679 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a6deada879885dca6c4b89e4a7394b55/cf1 in testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,679 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): a6deada879885dca6c4b89e4a7394b55/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:46,679 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a6deada879885dca6c4b89e4a7394b55/cf2 in testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,679 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91-top] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/.tmp, totalSize=34.9 K 2023-02-13 04:54:46,679 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde-top] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/.tmp, totalSize=34.9 K 2023-02-13 04:54:46,680 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264081127 2023-02-13 04:54:46,680 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264081127 2023-02-13 04:54:46,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:46,681 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264081319 2023-02-13 04:54:46,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,681 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264081319 2023-02-13 04:54:46,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cc7e469ee5eb3889f5d7044238390067, NAME => 'testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 04:54:46,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:46,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=a6deada879885dca6c4b89e4a7394b55, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:46,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:46,682 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264086681"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264086681"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264086681"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264086681"}]},"ts":"1676264086681"} 2023-02-13 04:54:46,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,682 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264081526 2023-02-13 04:54:46,682 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264081526 2023-02-13 04:54:46,683 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264081734 2023-02-13 04:54:46,683 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264081734 2023-02-13 04:54:46,683 INFO [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,684 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676264081942 2023-02-13 04:54:46,684 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676264081942 2023-02-13 04:54:46,684 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676264082140 2023-02-13 04:54:46,685 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676264082140 2023-02-13 04:54:46,685 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1 2023-02-13 04:54:46,685 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1 2023-02-13 04:54:46,685 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264082338 2023-02-13 04:54:46,685 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264082338 2023-02-13 04:54:46,687 INFO [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc7e469ee5eb3889f5d7044238390067 columnFamilyName cf1 2023-02-13 04:54:46,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-13 04:54:46,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure a6deada879885dca6c4b89e4a7394b55, server=jenkins-hbase4.apache.org,33859,1676264072829 in 254 msec 2023-02-13 04:54:46,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, ASSIGN in 424 msec 2023-02-13 04:54:46,709 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): a6deada879885dca6c4b89e4a7394b55#cf2#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:46,709 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d-bottom 2023-02-13 04:54:46,714 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445-bottom 2023-02-13 04:54:46,725 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): a6deada879885dca6c4b89e4a7394b55#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:46,737 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91-bottom 2023-02-13 04:54:46,744 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22-bottom 2023-02-13 04:54:46,761 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f-bottom 2023-02-13 04:54:46,774 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/.tmp/cf2/70264ed512684ed29f7be187bed9b980 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/70264ed512684ed29f7be187bed9b980 2023-02-13 04:54:46,781 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b-bottom 2023-02-13 04:54:46,781 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a6deada879885dca6c4b89e4a7394b55/cf2 of a6deada879885dca6c4b89e4a7394b55 into 70264ed512684ed29f7be187bed9b980(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:46,781 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6deada879885dca6c4b89e4a7394b55: 2023-02-13 04:54:46,781 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55., storeName=a6deada879885dca6c4b89e4a7394b55/cf2, priority=9, startTime=1676264086678; duration=0sec 2023-02-13 04:54:46,781 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:46,787 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623-bottom 2023-02-13 04:54:46,788 INFO [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(310): Store=cc7e469ee5eb3889f5d7044238390067/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:46,788 INFO [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,789 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2 2023-02-13 04:54:46,789 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2 2023-02-13 04:54:46,790 INFO [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc7e469ee5eb3889f5d7044238390067 columnFamilyName cf2 2023-02-13 04:54:46,795 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/.tmp/cf1/4220a9ea83c843ca8294f40b28d54e9c as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/4220a9ea83c843ca8294f40b28d54e9c 2023-02-13 04:54:46,802 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a6deada879885dca6c4b89e4a7394b55/cf1 of a6deada879885dca6c4b89e4a7394b55 into 4220a9ea83c843ca8294f40b28d54e9c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:46,803 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6deada879885dca6c4b89e4a7394b55: 2023-02-13 04:54:46,803 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55., storeName=a6deada879885dca6c4b89e4a7394b55/cf1, priority=9, startTime=1676264086677; duration=0sec 2023-02-13 04:54:46,803 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:46,810 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816-bottom 2023-02-13 04:54:46,814 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2-bottom 2023-02-13 04:54:46,819 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde-bottom 2023-02-13 04:54:46,824 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b-bottom 2023-02-13 04:54:46,827 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7-bottom 2023-02-13 04:54:46,834 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433-bottom 2023-02-13 04:54:46,839 DEBUG [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a-bottom 2023-02-13 04:54:46,839 INFO [StoreOpener-cc7e469ee5eb3889f5d7044238390067-1] regionserver.HStore(310): Store=cc7e469ee5eb3889f5d7044238390067/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:46,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:46,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:46,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cc7e469ee5eb3889f5d7044238390067; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61604560, jitterRate=-0.08202052116394043}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:46,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cc7e469ee5eb3889f5d7044238390067: Running coprocessor pre-open hook at 1676264086682Writing region info on filesystem at 1676264086682Initializing all the Stores at 1676264086683 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264086683Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264086683Cleaning up temporary data from old regions at 1676264086842 (+159 ms)Cleaning up detritus from prior splits at 1676264086843 (+1 ms)Running coprocessor post-open hooks at 1676264086846 (+3 ms)Region opened successfully at 1676264086846 2023-02-13 04:54:46,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067., pid=33, masterSystemTime=1676264086575 2023-02-13 04:54:46,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:46,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:46,848 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:46,848 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:46,848 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,848 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): cc7e469ee5eb3889f5d7044238390067/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:46,848 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,848 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cc7e469ee5eb3889f5d7044238390067/cf1 in testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,848 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): cc7e469ee5eb3889f5d7044238390067/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:46,848 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of cc7e469ee5eb3889f5d7044238390067/cf2 in testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,848 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/.tmp, totalSize=34.9 K 2023-02-13 04:54:46,849 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/.tmp, totalSize=34.9 K 2023-02-13 04:54:46,849 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264081127 2023-02-13 04:54:46,849 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264081127 2023-02-13 04:54:46,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:46,850 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=cc7e469ee5eb3889f5d7044238390067, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:46,850 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264081319 2023-02-13 04:54:46,850 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264081319 2023-02-13 04:54:46,851 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264086850"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264086850"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264086850"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264086850"}]},"ts":"1676264086850"} 2023-02-13 04:54:46,851 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264081526 2023-02-13 04:54:46,851 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264081526 2023-02-13 04:54:46,851 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264081734 2023-02-13 04:54:46,852 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264081734 2023-02-13 04:54:46,852 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264081942 2023-02-13 04:54:46,852 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264081942 2023-02-13 04:54:46,853 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264082140 2023-02-13 04:54:46,853 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264082140 2023-02-13 04:54:46,853 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264082338 2023-02-13 04:54:46,853 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264082338 2023-02-13 04:54:46,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-13 04:54:46,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure cc7e469ee5eb3889f5d7044238390067, server=jenkins-hbase4.apache.org,33859,1676264072829 in 428 msec 2023-02-13 04:54:46,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 04:54:46,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, ASSIGN in 592 msec 2023-02-13 04:54:46,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 in 4.2570 sec 2023-02-13 04:54:46,881 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): cc7e469ee5eb3889f5d7044238390067#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:46,893 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): cc7e469ee5eb3889f5d7044238390067#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:46,928 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/.tmp/cf2/1396695259624fb9a18299aae5345a90 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1396695259624fb9a18299aae5345a90 2023-02-13 04:54:46,929 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/.tmp/cf1/de6a822ed0aa4e97a22ca18531fa6778 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/de6a822ed0aa4e97a22ca18531fa6778 2023-02-13 04:54:46,935 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cc7e469ee5eb3889f5d7044238390067/cf2 of cc7e469ee5eb3889f5d7044238390067 into 1396695259624fb9a18299aae5345a90(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:46,935 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for cc7e469ee5eb3889f5d7044238390067: 2023-02-13 04:54:46,935 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067., storeName=cc7e469ee5eb3889f5d7044238390067/cf2, priority=9, startTime=1676264086848; duration=0sec 2023-02-13 04:54:46,935 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:46,949 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cc7e469ee5eb3889f5d7044238390067/cf1 of cc7e469ee5eb3889f5d7044238390067 into de6a822ed0aa4e97a22ca18531fa6778(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:46,949 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cc7e469ee5eb3889f5d7044238390067: 2023-02-13 04:54:46,949 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067., storeName=cc7e469ee5eb3889f5d7044238390067/cf1, priority=9, startTime=1676264086847; duration=0sec 2023-02-13 04:54:46,949 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:47,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33859] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:50508 deadline: 1676264147007, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94. is not online on jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:47,122 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:54:47,130 INFO [Listener at localhost/33551] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 04:54:47,132 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:54:47,149 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:47,151 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 04:54:47,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-13 04:54:47,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 04:54:47,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264087156"}]},"ts":"1676264087156"} 2023-02-13 04:54:47,157 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 04:54:47,159 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 04:54:47,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, UNASSIGN}] 2023-02-13 04:54:47,161 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, UNASSIGN 2023-02-13 04:54:47,161 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=a6deada879885dca6c4b89e4a7394b55, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:47,162 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264087161"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264087161"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264087161"}]},"ts":"1676264087161"} 2023-02-13 04:54:47,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure a6deada879885dca6c4b89e4a7394b55, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:47,166 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, UNASSIGN 2023-02-13 04:54:47,166 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=cc7e469ee5eb3889f5d7044238390067, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:47,167 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264087166"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264087166"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264087166"}]},"ts":"1676264087166"} 2023-02-13 04:54:47,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure cc7e469ee5eb3889f5d7044238390067, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:54:47,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 04:54:47,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:47,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a6deada879885dca6c4b89e4a7394b55, disabling compactions & flushes 2023-02-13 04:54:47,316 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:47,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:47,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. after waiting 0 ms 2023-02-13 04:54:47,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:47,330 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91-top] to archive 2023-02-13 04:54:47,331 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:47,333 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,335 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,336 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,337 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,338 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,340 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,341 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,343 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde-top] to archive 2023-02-13 04:54:47,344 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:47,345 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,347 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,348 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,349 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,350 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,351 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,352 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:54:47,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:47,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55. 2023-02-13 04:54:47,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a6deada879885dca6c4b89e4a7394b55: Waiting for close lock at 1676264087315Running coprocessor pre-close hooks at 1676264087315Disabling compacts and flushes for region at 1676264087315Disabling writes for close at 1676264087316 (+1 ms)Writing region close event to WAL at 1676264087353 (+37 ms)Running coprocessor post-close hooks at 1676264087357 (+4 ms)Closed at 1676264087357 2023-02-13 04:54:47,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:47,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:47,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cc7e469ee5eb3889f5d7044238390067, disabling compactions & flushes 2023-02-13 04:54:47,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:47,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:47,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. after waiting 0 ms 2023-02-13 04:54:47,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:47,359 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=a6deada879885dca6c4b89e4a7394b55, regionState=CLOSED 2023-02-13 04:54:47,359 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264087359"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264087359"}]},"ts":"1676264087359"} 2023-02-13 04:54:47,360 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91-bottom] to archive 2023-02-13 04:54:47,361 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:47,363 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 04:54:47,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure a6deada879885dca6c4b89e4a7394b55, server=jenkins-hbase4.apache.org,33859,1676264072829 in 198 msec 2023-02-13 04:54:47,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a6deada879885dca6c4b89e4a7394b55, UNASSIGN in 203 msec 2023-02-13 04:54:47,365 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,366 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,367 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,369 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,370 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,372 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,373 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde-bottom] to archive 2023-02-13 04:54:47,374 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:47,376 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,377 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,378 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,379 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,380 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,381 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,382 DEBUG [StoreCloser-testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:54:47,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:47,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067. 2023-02-13 04:54:47,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cc7e469ee5eb3889f5d7044238390067: Waiting for close lock at 1676264087359Running coprocessor pre-close hooks at 1676264087359Disabling compacts and flushes for region at 1676264087359Disabling writes for close at 1676264087359Writing region close event to WAL at 1676264087384 (+25 ms)Running coprocessor post-close hooks at 1676264087387 (+3 ms)Closed at 1676264087387 2023-02-13 04:54:47,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:47,389 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=cc7e469ee5eb3889f5d7044238390067, regionState=CLOSED 2023-02-13 04:54:47,390 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264087389"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264087389"}]},"ts":"1676264087389"} 2023-02-13 04:54:47,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 04:54:47,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure cc7e469ee5eb3889f5d7044238390067, server=jenkins-hbase4.apache.org,33859,1676264072829 in 223 msec 2023-02-13 04:54:47,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 04:54:47,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cc7e469ee5eb3889f5d7044238390067, UNASSIGN in 234 msec 2023-02-13 04:54:47,397 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264087397"}]},"ts":"1676264087397"} 2023-02-13 04:54:47,399 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 04:54:47,400 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 04:54:47,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 249 msec 2023-02-13 04:54:47,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 04:54:47,458 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 04:54:47,459 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-13 04:54:47,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,463 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 04:54:47,470 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:47,470 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:47,470 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,472 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/recovered.edits] 2023-02-13 04:54:47,473 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/recovered.edits] 2023-02-13 04:54:47,473 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/recovered.edits] 2023-02-13 04:54:47,476 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/.regioninfo to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/.regioninfo 2023-02-13 04:54:47,476 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/1bd160361bfb4bffa7d04988f01beb7d 2023-02-13 04:54:47,477 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/.regioninfo to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/.regioninfo 2023-02-13 04:54:47,478 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/3152b757c29544aa9b6496d4fbda6445 2023-02-13 04:54:47,480 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/53291da7d9a242ee8bfb1e7ee6453b91 2023-02-13 04:54:47,480 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,481 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/1bd160361bfb4bffa7d04988f01beb7d.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,482 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/55d71b080bf44c4a951c1fd701770f22 2023-02-13 04:54:47,482 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,483 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/3152b757c29544aa9b6496d4fbda6445.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,484 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/587a1600f61e487eb6eef39085e1250f 2023-02-13 04:54:47,484 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,484 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/53291da7d9a242ee8bfb1e7ee6453b91.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,486 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/981c8d1730324863bf7f6e8d267d100b 2023-02-13 04:54:47,486 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,486 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/55d71b080bf44c4a951c1fd701770f22.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,487 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf1/d88fb96b4e0547b9adf5b5d1db626623 2023-02-13 04:54:47,487 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,488 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/587a1600f61e487eb6eef39085e1250f.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,489 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,489 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/981c8d1730324863bf7f6e8d267d100b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,490 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,491 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/08fd259382374a54a96419d51b051816 2023-02-13 04:54:47,491 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf1/d88fb96b4e0547b9adf5b5d1db626623.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,492 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/1b7c7c77807b47debc75fd9183ecabd2 2023-02-13 04:54:47,494 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/3bfa7c4b8e84456580b6fd5621dd1dde 2023-02-13 04:54:47,495 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,496 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b 2023-02-13 04:54:47,496 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/08fd259382374a54a96419d51b051816.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,497 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,498 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/1b7c7c77807b47debc75fd9183ecabd2.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,498 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9b28c5125c95434e931019ddf2923ee7 2023-02-13 04:54:47,499 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,500 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/9bfa185affaa4d878c1cacd81770c433 2023-02-13 04:54:47,501 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/3bfa7c4b8e84456580b6fd5621dd1dde.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,501 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,502 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/cf2/ada18712032345cda6af2c7e9af5683a 2023-02-13 04:54:47,503 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,503 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/69f678bb5e444cbbb0eeb6ef0872ea3b.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,505 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,505 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/9b28c5125c95434e931019ddf2923ee7.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,507 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/recovered.edits/85.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94/recovered.edits/85.seqid 2023-02-13 04:54:47,507 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cc7e469ee5eb3889f5d7044238390067/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,507 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/9bfa185affaa4d878c1cacd81770c433.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,508 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,512 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/a6deada879885dca6c4b89e4a7394b55/cf2/ada18712032345cda6af2c7e9af5683a.8ad8f350a2b6f56d4128dc02f389ea94 2023-02-13 04:54:47,513 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/de6a822ed0aa4e97a22ca18531fa6778 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf1/de6a822ed0aa4e97a22ca18531fa6778 2023-02-13 04:54:47,515 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/4220a9ea83c843ca8294f40b28d54e9c to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf1/4220a9ea83c843ca8294f40b28d54e9c 2023-02-13 04:54:47,515 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1396695259624fb9a18299aae5345a90 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/cf2/1396695259624fb9a18299aae5345a90 2023-02-13 04:54:47,517 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/70264ed512684ed29f7be187bed9b980 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/cf2/70264ed512684ed29f7be187bed9b980 2023-02-13 04:54:47,519 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067/recovered.edits/90.seqid 2023-02-13 04:54:47,522 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/cc7e469ee5eb3889f5d7044238390067 2023-02-13 04:54:47,523 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55/recovered.edits/90.seqid 2023-02-13 04:54:47,524 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitWithoutPONR/a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:47,524 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 04:54:47,526 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,529 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 04:54:47,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 04:54:47,533 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,533 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 04:54:47,533 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264087533"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:47,533 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264087533"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:47,533 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264087533"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:47,535 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:54:47,535 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 8ad8f350a2b6f56d4128dc02f389ea94, NAME => 'testSplitWithoutPONR,,1676264080121.8ad8f350a2b6f56d4128dc02f389ea94.', STARTKEY => '', ENDKEY => ''}, {ENCODED => cc7e469ee5eb3889f5d7044238390067, NAME => 'testSplitWithoutPONR,,1676264082602.cc7e469ee5eb3889f5d7044238390067.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => a6deada879885dca6c4b89e4a7394b55, NAME => 'testSplitWithoutPONR,41,1676264082602.a6deada879885dca6c4b89e4a7394b55.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 04:54:47,535 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 04:54:47,535 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264087535"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:47,537 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 04:54:47,539 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:47,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 81 msec 2023-02-13 04:54:47,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 04:54:47,565 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 04:54:47,583 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=401 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35864 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:39058 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:53616 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:53638 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35866 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:55674 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35894 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35914 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fc298c7-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:39100 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:55632 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:34618 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35856 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:39108 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:33859-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35898 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:39068 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6fc298c7-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_141819295_15 at /127.0.0.1:35882 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=859 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=249 (was 250), ProcessCount=168 (was 168), AvailableMemoryMB=3757 (was 3886) 2023-02-13 04:54:47,601 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=401, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=249, ProcessCount=168, AvailableMemoryMB=3757 2023-02-13 04:54:47,601 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:47,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:54:47,605 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:54:47,607 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:47,609 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:54:47,610 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:54:47,612 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,612 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 empty. 2023-02-13 04:54:47,613 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,613 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 04:54:47,625 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:47,626 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3bcdfe2dbd4e99a75b6c62f3d21619a6, NAME => 'testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:54:47,635 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:47,636 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 3bcdfe2dbd4e99a75b6c62f3d21619a6, disabling compactions & flushes 2023-02-13 04:54:47,636 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,636 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,636 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. after waiting 0 ms 2023-02-13 04:54:47,636 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,636 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,636 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: Waiting for close lock at 1676264087636Disabling compacts and flushes for region at 1676264087636Disabling writes for close at 1676264087636Writing region close event to WAL at 1676264087636Closed at 1676264087636 2023-02-13 04:54:47,638 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:54:47,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264087639"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264087639"}]},"ts":"1676264087639"} 2023-02-13 04:54:47,641 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:54:47,641 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:54:47,642 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264087641"}]},"ts":"1676264087641"} 2023-02-13 04:54:47,643 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 04:54:47,646 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:47,647 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:47,647 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:47,647 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:47,647 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:47,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN}] 2023-02-13 04:54:47,649 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:47,649 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=false 2023-02-13 04:54:47,800 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:47,801 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:47,801 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264087801"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264087801"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264087801"}]},"ts":"1676264087801"} 2023-02-13 04:54:47,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:47,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3bcdfe2dbd4e99a75b6c62f3d21619a6, NAME => 'testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:47,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:47,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:47,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,961 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,962 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 2023-02-13 04:54:47,962 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 2023-02-13 04:54:47,963 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bcdfe2dbd4e99a75b6c62f3d21619a6 columnFamilyName cf1 2023-02-13 04:54:47,963 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(310): Store=3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:47,963 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,964 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 2023-02-13 04:54:47,965 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 2023-02-13 04:54:47,965 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bcdfe2dbd4e99a75b6c62f3d21619a6 columnFamilyName cf2 2023-02-13 04:54:47,965 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(310): Store=3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:47,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:47,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:47,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:47,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3bcdfe2dbd4e99a75b6c62f3d21619a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73964883, jitterRate=0.10216264426708221}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:47,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: Running coprocessor pre-open hook at 1676264087960Writing region info on filesystem at 1676264087960Initializing all the Stores at 1676264087960Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264087961 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264087961Cleaning up temporary data from old regions at 1676264087967 (+6 ms)Cleaning up detritus from prior splits at 1676264087967Running coprocessor post-open hooks at 1676264087971 (+4 ms)Region opened successfully at 1676264087971 2023-02-13 04:54:47,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6., pid=43, masterSystemTime=1676264087955 2023-02-13 04:54:47,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:47,974 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:47,974 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264087974"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264087974"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264087974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264087974"}]},"ts":"1676264087974"} 2023-02-13 04:54:47,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 04:54:47,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 in 173 msec 2023-02-13 04:54:47,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 04:54:47,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN in 331 msec 2023-02-13 04:54:47,980 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:54:47,980 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264087980"}]},"ts":"1676264087980"} 2023-02-13 04:54:47,982 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 04:54:47,988 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:54:47,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 382 msec 2023-02-13 04:54:48,115 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:48,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 04:54:48,117 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,117 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,117 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 04:54:48,117 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:54:48,118 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,118 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,119 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,119 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,119 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,119 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,120 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,120 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,121 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,121 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,122 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:48,122 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,122 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. started... 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,122 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,123 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:54:48,123 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,123 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,124 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,124 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,125 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,125 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,138 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/c50debe93c124539a0b6b1eb31da10cf 2023-02-13 04:54:48,152 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/51e47f2e2793464cb165b9a079cb5151 2023-02-13 04:54:48,156 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/c50debe93c124539a0b6b1eb31da10cf as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf 2023-02-13 04:54:48,160 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:48,161 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/51e47f2e2793464cb165b9a079cb5151 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 2023-02-13 04:54:48,165 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:48,166 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 43ms, sequenceid=6, compaction requested=false 2023-02-13 04:54:48,166 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,167 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:48,167 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,167 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,169 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,169 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,170 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,170 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 04:54:48,172 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:54:48,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54fb1cab[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:48,172 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:48,173 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:48,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:48,175 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,175 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,175 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,175 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:48,181 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:54:48,181 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 04:54:48,181 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:48,181 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:48,181 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 04:54:48,181 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,184 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,183 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,184 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,185 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,187 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,187 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,187 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,191 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,191 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,192 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,292 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:48,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:48,310 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:48,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:48,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 04:54:48,312 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,312 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,312 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 04:54:48,312 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:54:48,312 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,312 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,314 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,314 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,314 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,314 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,315 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,315 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,316 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,316 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,316 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,317 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,317 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,317 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,317 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,317 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,317 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,317 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. started... 2023-02-13 04:54:48,318 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:48,320 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,321 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,322 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,322 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,322 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,322 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,332 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/aa6cd264297142efa2249c453e57aa43 2023-02-13 04:54:48,345 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/a6c2661ae8944403af4d99c86e942fd1 2023-02-13 04:54:48,350 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/aa6cd264297142efa2249c453e57aa43 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 2023-02-13 04:54:48,355 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:48,355 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/a6c2661ae8944403af4d99c86e942fd1 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 2023-02-13 04:54:48,360 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:48,361 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 43ms, sequenceid=19, compaction requested=false 2023-02-13 04:54:48,361 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:48,361 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,361 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:48,361 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:48,361 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,361 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,361 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,363 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 04:54:48,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cd54308[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:48,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:54:48,366 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,367 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,367 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:48,367 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,367 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:48,368 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,368 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:48,368 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,368 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,368 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:48,370 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,370 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,370 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,377 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:48,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:54:48,381 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 04:54:48,381 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:48,381 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:48,381 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 04:54:48,381 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,383 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,384 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,385 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,386 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,387 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,387 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,391 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,391 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,391 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,391 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,392 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,492 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:48,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:48,509 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:48,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:48,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 04:54:48,510 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,510 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 04:54:48,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:54:48,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,513 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,513 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,513 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,514 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,515 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,515 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:48,515 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:48,516 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,516 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. started... 2023-02-13 04:54:48,516 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:48,517 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,517 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,517 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,517 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,517 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,517 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,525 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/42209a1628054e68adcfb5ce2018e888 2023-02-13 04:54:48,547 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/7988f443f32b437f8880beca3ee71ab8 2023-02-13 04:54:48,551 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/42209a1628054e68adcfb5ce2018e888 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 2023-02-13 04:54:48,555 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:48,556 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/7988f443f32b437f8880beca3ee71ab8 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 2023-02-13 04:54:48,559 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:48,560 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 44ms, sequenceid=32, compaction requested=true 2023-02-13 04:54:48,561 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:48,561 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,561 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:48,561 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:48,561 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,561 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,561 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,563 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,563 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,563 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,563 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 04:54:48,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@777ee6b5[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:48,565 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:54:48,565 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,567 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,567 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,567 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,568 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,568 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:48,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:48,568 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,571 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,571 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,571 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:48,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:54:48,575 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 04:54:48,575 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:48,575 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:48,575 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 04:54:48,575 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:48,576 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,576 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,577 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,578 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,579 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,580 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,581 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,581 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,587 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,587 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,587 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,588 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:48,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,588 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:48,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,587 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,588 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,588 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,688 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:48,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:48,705 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:48,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:48,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 04:54:48,706 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,706 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,707 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 04:54:48,707 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:54:48,707 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,707 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,708 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,708 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,708 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,708 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,709 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,709 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:48,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:48,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:48,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,710 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,711 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,711 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,711 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. started... 2023-02-13 04:54:48,711 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:48,712 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,712 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,713 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,713 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,713 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,713 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,721 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/6701697f5dfa431ca736d6551776a361 2023-02-13 04:54:48,737 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/eef0dc8cd3ae486d92de9a504660c7ab 2023-02-13 04:54:48,742 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/6701697f5dfa431ca736d6551776a361 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 2023-02-13 04:54:48,747 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:48,748 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/eef0dc8cd3ae486d92de9a504660c7ab as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab 2023-02-13 04:54:48,753 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:48,754 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 43ms, sequenceid=45, compaction requested=true 2023-02-13 04:54:48,754 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:48,754 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,754 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,756 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,756 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 04:54:48,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b9dd5fd[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:48,758 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:54:48,758 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,760 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:48,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,760 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,760 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:48,761 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,761 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,761 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:48,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,761 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,762 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:48,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:54:48,766 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 04:54:48,766 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:48,766 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:48,766 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 04:54:48,766 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:48,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,768 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,768 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,769 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,769 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,770 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,771 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,771 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,774 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,774 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:48,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,774 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,774 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,774 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:48,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,774 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,774 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:48,774 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,775 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,875 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:48,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:48,891 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:48,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:48,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 04:54:48,893 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,893 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,893 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 04:54:48,893 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:54:48,894 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,894 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,896 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,896 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,896 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,896 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,896 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,896 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,897 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,897 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,897 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,898 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:48,898 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:48,898 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,898 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. started... 2023-02-13 04:54:48,898 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:48,899 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,900 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,900 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,900 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,900 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,900 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,914 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/effc55ad0971459ab3f9801ba067e3e0 2023-02-13 04:54:48,931 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/aff02efde588496ba6217076c9a5d8b7 2023-02-13 04:54:48,936 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/effc55ad0971459ab3f9801ba067e3e0 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 2023-02-13 04:54:48,940 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:48,941 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/aff02efde588496ba6217076c9a5d8b7 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 2023-02-13 04:54:48,946 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:48,947 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 49ms, sequenceid=58, compaction requested=true 2023-02-13 04:54:48,947 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:48,947 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:48,947 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:48,947 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:48,947 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,947 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:48,947 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:48,949 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,949 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,949 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,949 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:48,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 04:54:48,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@278f8a61[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:48,951 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:54:48,951 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,953 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,953 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,953 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,954 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,954 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:48,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:48,954 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:48,955 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,955 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,956 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,956 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,960 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,961 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:48,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:48,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:54:48,963 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 04:54:48,963 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:48,963 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:48,963 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 04:54:48,964 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:48,966 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,967 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,967 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,966 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:48,967 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:48,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,968 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:48,968 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:48,970 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:48,970 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,971 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 04:54:48,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:48,973 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:48,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,973 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:48,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:48,974 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:48,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:48,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,074 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:49,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:49,090 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:49,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:49,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 04:54:49,092 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 04:54:49,092 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:49,092 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 04:54:49,092 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:54:49,092 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,092 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,094 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:49,094 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:49,094 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:49,094 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:49,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:49,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:49,094 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,094 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:49,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:49,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:49,096 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:49,096 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:49,096 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:49,096 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. started... 2023-02-13 04:54:49,096 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:49,096 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:49,097 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,099 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,099 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,099 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:49,100 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,100 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:49,106 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/001cab63e8834d61b300afb089f0b200 2023-02-13 04:54:49,118 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/c9a55339a3fb4abdab2ffb719068620e 2023-02-13 04:54:49,122 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/001cab63e8834d61b300afb089f0b200 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 2023-02-13 04:54:49,126 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:49,127 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/c9a55339a3fb4abdab2ffb719068620e as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e 2023-02-13 04:54:49,131 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:49,132 INFO [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 36ms, sequenceid=71, compaction requested=true 2023-02-13 04:54:49,132 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:49,132 DEBUG [rs(jenkins-hbase4.apache.org,46601,1676264072787)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:49,132 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:49,132 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:49,132 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,132 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 04:54:49,132 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 04:54:49,134 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,134 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:49,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:49,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 04:54:49,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4124409b[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:49,136 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:54:49,136 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:49,137 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,137 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:49,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:49,137 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:49,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:54:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 04:54:49,138 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:49,139 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,139 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:49,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:49,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:49,139 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:49,139 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 04:54:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:49,141 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 04:54:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:49,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46601,1676264072787': 2023-02-13 04:54:49,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:54:49,144 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 04:54:49,144 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:49,144 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:49,144 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 04:54:49,144 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:49,145 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,145 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:49,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,145 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,145 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:49,146 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:49,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:49,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:49,146 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,146 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:49,147 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:49,147 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:49,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:49,148 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 04:54:49,149 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,149 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:49,155 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:49,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:49,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:49,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:49,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:49,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:49,155 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 04:54:49,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:49,155 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:49,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:49,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:49,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 04:54:49,156 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 04:54:49,256 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:49,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:49,517 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 04:54:49,518 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:49,520 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,521 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,521 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,521 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:49,522 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:49,522 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:49,770 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,770 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:49,770 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:49,770 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:49,770 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 04:54:49,770 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:49,770 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:49,771 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:49,771 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:49,771 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:49,772 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 04:54:49,773 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:49,773 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:49,773 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:49,773 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:49,774 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:49,774 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:49,774 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:49,774 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:49,774 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:49,775 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:49,775 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:49,775 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:49,775 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:49,776 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,776 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:49,776 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,776 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,776 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,776 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:49,776 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:49,777 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:49,777 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264119777 2023-02-13 04:54:49,777 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:49,779 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,782 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:49,782 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:49,782 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:49,782 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264149782 2023-02-13 04:54:49,782 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264209782 2023-02-13 04:54:49,782 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 04:54:49,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:49,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:49,786 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:49,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:49,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:50,033 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:50,033 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:50,034 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:50,035 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 04:54:50,035 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:50,036 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:50,036 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:50,036 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:50,037 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,037 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:50,037 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,037 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,038 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,038 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:50,038 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,038 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,039 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:50,039 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,039 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,039 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,040 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,040 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,040 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:50,041 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:50,041 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:50,041 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264120041 2023-02-13 04:54:50,042 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:50,043 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,045 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:50,045 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:50,045 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:50,045 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264150045 2023-02-13 04:54:50,045 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264210045 2023-02-13 04:54:50,046 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:50,046 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:50,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN}] 2023-02-13 04:54:50,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:50,048 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:50,048 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:50,296 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,296 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:50,296 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:50,296 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:50,296 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:50,296 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:50,296 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:50,297 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:50,297 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:50,297 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:50,299 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 04:54:50,299 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:50,300 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:50,300 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:50,300 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:50,300 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,301 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:50,301 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,301 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,301 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,302 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:50,302 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,303 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,303 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:50,304 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,304 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,304 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,304 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,304 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,304 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:54:50,306 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:50,307 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:50,307 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264120307 2023-02-13 04:54:50,308 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:50,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,311 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:50,311 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:50,311 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:50,311 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264150311 2023-02-13 04:54:50,311 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264210311 2023-02-13 04:54:50,311 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:50,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN}] 2023-02-13 04:54:50,312 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:50,313 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,314 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:50,314 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264090314"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264090314"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264090314"}]},"ts":"1676264090314"} 2023-02-13 04:54:50,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:50,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:50,316 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:50,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:50,561 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,562 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:50,562 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:50,562 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:50,564 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:50,564 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:50,564 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:50,564 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:50,564 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:50,565 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:50,566 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:50,566 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 04:54:50,566 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:50,567 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:50,567 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:50,567 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:50,567 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,567 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:50,568 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,568 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,568 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,568 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:50,568 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,569 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,569 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:50,569 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,570 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,570 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,570 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,570 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,570 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,570 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,570 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,570 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:50,571 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:50,571 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:50,571 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:50,571 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264120571 2023-02-13 04:54:50,572 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:50,572 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,574 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:50,574 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:50,574 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:50,574 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264150574 2023-02-13 04:54:50,575 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264210575 2023-02-13 04:54:50,575 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:50,575 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:50,575 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264090575"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264090575"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264090575"}]},"ts":"1676264090575"} 2023-02-13 04:54:50,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:50,576 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:50,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:50,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3bcdfe2dbd4e99a75b6c62f3d21619a6, disabling compactions & flushes 2023-02-13 04:54:50,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:50,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:50,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. after waiting 0 ms 2023-02-13 04:54:50,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:50,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:54:50,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/00d8e7b54cf143a0a5f6a4741947c274 2023-02-13 04:54:50,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/3bdbf0b48c2844f996af19c084bc6f00 2023-02-13 04:54:50,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/00d8e7b54cf143a0a5f6a4741947c274 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 2023-02-13 04:54:50,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:54:50,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/3bdbf0b48c2844f996af19c084bc6f00 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 2023-02-13 04:54:50,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:54:50,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 39ms, sequenceid=82, compaction requested=true 2023-02-13 04:54:50,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:54:50,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:50,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:50,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: Waiting for close lock at 1676264090729Running coprocessor pre-close hooks at 1676264090729Disabling compacts and flushes for region at 1676264090729Disabling writes for close at 1676264090729Obtaining lock to block concurrent updates at 1676264090729Preparing flush snapshotting stores in 3bcdfe2dbd4e99a75b6c62f3d21619a6 at 1676264090729Finished memstore snapshotting testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676264090730 (+1 ms)Flushing stores of testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. at 1676264090730Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1: creating writer at 1676264090731 (+1 ms)Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1: appending metadata at 1676264090733 (+2 ms)Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1: closing flushed file at 1676264090733Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2: creating writer at 1676264090744 (+11 ms)Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2: appending metadata at 1676264090746 (+2 ms)Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2: closing flushed file at 1676264090746Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1: reopening flushed file at 1676264090759 (+13 ms)Flushing 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2: reopening flushed file at 1676264090764 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3bcdfe2dbd4e99a75b6c62f3d21619a6 in 39ms, sequenceid=82, compaction requested=true at 1676264090768 (+4 ms)Writing region close event to WAL at 1676264090770 (+2 ms)Running coprocessor post-close hooks at 1676264090773 (+3 ms)Closed at 1676264090773 2023-02-13 04:54:50,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:50,776 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED 2023-02-13 04:54:50,776 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264090775"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264090775"}]},"ts":"1676264090775"} 2023-02-13 04:54:50,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:50,777 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:50,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:50,777 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:50,825 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,825 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:50,825 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:50,825 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:50,826 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:50,826 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:50,826 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:50,826 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:50,826 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:50,827 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:50,827 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:50,827 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 04:54:50,828 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 04:54:50,828 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:50,828 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:50,828 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:50,828 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:50,829 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,829 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:50,829 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,829 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:50,830 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,830 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:50,830 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,830 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:50,830 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:50,831 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,831 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,831 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:50,831 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,831 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,831 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,831 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:50,831 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,831 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,832 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:50,832 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:50,832 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:50,833 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:50,833 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:50,836 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264120836 2023-02-13 04:54:50,837 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:50,837 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:50,840 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:50,840 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:50,840 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:50,840 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264150840 2023-02-13 04:54:50,840 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264210840 2023-02-13 04:54:50,840 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:54:50,840 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:50,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264090840"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264090840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264090840"}]},"ts":"1676264090840"} 2023-02-13 04:54:50,842 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:50,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 04:54:50,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 in 266 msec 2023-02-13 04:54:50,845 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:50,845 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:50,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:50,845 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:51,090 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,091 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:51,091 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:51,091 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:51,092 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:51,092 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:51,092 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:51,092 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:51,092 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:51,093 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:51,094 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:51,094 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 04:54:51,094 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:51,094 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:51,095 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:51,095 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:51,095 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:51,095 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,096 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:51,096 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,096 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,096 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,096 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:51,097 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,097 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,097 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:51,098 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,098 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:51,098 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:51,098 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:51,098 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,098 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,098 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,098 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,098 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,098 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:51,099 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:51,099 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:51,100 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:51,100 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264121100 2023-02-13 04:54:51,100 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:51,101 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:51,103 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:51,103 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:51,103 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:51,103 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264151103 2023-02-13 04:54:51,104 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264211104 2023-02-13 04:54:51,104 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:51,104 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:51,105 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:51,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 04:54:51,108 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN in 793 msec 2023-02-13 04:54:51,111 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:51,111 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:51,111 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:51,354 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,354 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:51,354 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:51,354 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:51,355 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:51,355 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:51,355 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:51,355 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:51,356 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:51,356 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:51,357 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:51,357 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:51,357 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:51,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:51,358 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:51,358 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:51,358 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:51,358 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,359 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:51,359 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,359 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,359 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,360 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:51,360 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,360 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,360 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:51,361 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,361 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:51,361 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:51,361 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,361 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,361 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,361 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:51,362 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:51,362 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:51,362 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264121362 2023-02-13 04:54:51,362 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:51,364 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,366 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:51,366 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:51,366 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:51,366 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264151366 2023-02-13 04:54:51,366 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264211366 2023-02-13 04:54:51,367 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:51,367 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,367 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,367 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:51,382 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, threads=14 2023-02-13 04:54:51,383 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,383 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,384 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,384 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,384 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,384 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,384 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,384 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,385 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,385 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,386 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,386 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,386 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,386 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,495 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,507 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,509 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,520 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,520 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,523 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,523 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,524 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,525 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,525 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,528 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,536 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,544 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,546 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,546 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 3bcdfe2dbd4e99a75b6c62f3d21619a6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:54:51,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:51,588 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:51,588 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:51,617 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:51,617 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:51,618 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:51,619 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 04:54:51,619 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:51,619 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:51,620 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:51,620 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:51,621 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:51,621 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:51,621 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,621 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:51,621 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,621 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,622 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,622 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:51,622 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,622 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,623 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:51,623 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,624 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:51,624 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:51,624 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:51,624 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,624 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,624 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,624 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,625 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,625 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:54:51,625 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:51,626 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:51,626 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264121626 2023-02-13 04:54:51,628 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,628 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:51,631 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:51,631 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:51,631 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:51,631 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264151631 2023-02-13 04:54:51,631 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264211631 2023-02-13 04:54:51,631 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:51,640 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, threads=14 2023-02-13 04:54:51,641 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,641 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,641 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,642 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,643 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,643 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,643 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,766 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,766 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,772 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,777 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,779 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,780 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,788 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,794 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,795 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,802 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,802 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,802 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,802 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,805 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab for region: 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:51,805 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 3bcdfe2dbd4e99a75b6c62f3d21619a6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:54:51,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:51,847 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/84e3a6e0598265f86960fec92a925b18/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:54:51,848 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/a7f3a8cf991d76c5fb7b296292b15b64/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:54:51,848 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:51,848 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:51,848 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:51,882 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,882 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:51,882 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:51,882 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:51,883 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:51,883 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:51,883 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:51,883 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:51,883 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:51,884 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:51,885 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 04:54:51,885 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:51,885 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:51,886 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:51,886 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:51,886 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:51,887 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:51,887 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,887 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:51,888 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,888 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:51,888 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,889 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:51,889 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,889 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:51,889 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:51,890 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,890 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:51,890 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:51,892 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:51,892 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,893 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,893 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,893 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,893 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:51,893 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 04:54:51,894 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:51,894 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:51,895 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264121895 2023-02-13 04:54:51,896 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:51,896 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,898 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:51,898 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:51,898 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:51,899 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264151899 2023-02-13 04:54:51,899 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264211899 2023-02-13 04:54:51,899 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:54:51,901 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:51,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:51,904 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:51,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:51,904 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:52,081 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:54:52,149 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:52,150 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:52,150 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:52,151 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:52,152 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 04:54:52,152 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:52,152 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:52,153 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:52,153 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:52,153 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:52,153 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:52,154 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,154 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:52,154 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,154 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,154 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,155 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:52,155 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,155 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,155 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:52,155 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,156 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:52,156 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:52,156 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:52,156 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,156 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,156 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,156 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,156 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,156 DEBUG [Listener at localhost/33551] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,156 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:52,157 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:52,157 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:52,158 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264122158 2023-02-13 04:54:52,160 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,160 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:52,163 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:52,164 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:52,164 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:52,164 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264152164 2023-02-13 04:54:52,164 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264212164 2023-02-13 04:54:52,164 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:52,165 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:52,178 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:52,179 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:52,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:52,179 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:52,415 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,415 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:52,415 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:52,415 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:52,416 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:52,416 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:52,416 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:52,416 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:52,416 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:52,417 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:52,418 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 04:54:52,418 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:52,419 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:52,419 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:52,419 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:52,420 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:52,420 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:52,420 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,420 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:52,421 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,421 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,421 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,421 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:52,421 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,422 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,422 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:52,423 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,423 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:52,423 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:52,423 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:52,424 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,424 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,424 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,424 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,424 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,424 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:54:52,425 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:52,425 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:52,425 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264122425 2023-02-13 04:54:52,427 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:52,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,430 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:52,430 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:52,430 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:52,430 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264152430 2023-02-13 04:54:52,431 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264212431 2023-02-13 04:54:52,431 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:52,431 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:52,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:52,433 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:52,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:52,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:52,682 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:52,682 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:52,683 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:52,684 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 04:54:52,684 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:52,684 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:52,684 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:52,685 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:52,685 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:52,685 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:52,685 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,685 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:52,686 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,686 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,686 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,686 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:52,686 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,686 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,687 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:52,687 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,687 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:52,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:52,688 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:52,688 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,688 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,688 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,688 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,688 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,688 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:52,689 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:52,689 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:52,689 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264122689 2023-02-13 04:54:52,690 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:52,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,693 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:52,694 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:52,694 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:52,694 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264152694 2023-02-13 04:54:52,694 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264212694 2023-02-13 04:54:52,694 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:52,694 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:52,696 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:52,696 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:52,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:52,696 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:52,944 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,944 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:52,944 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:52,944 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:52,945 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:52,945 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:52,945 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:52,945 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:52,945 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:52,946 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:52,946 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 04:54:52,946 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:52,946 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:52,947 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:52,947 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:52,947 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:52,947 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:52,948 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,948 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:52,948 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,948 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:52,948 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,949 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:52,949 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,949 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:52,949 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:52,950 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,950 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:52,950 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:52,950 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:52,950 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,950 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,950 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,950 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,950 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:52,950 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:52,951 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:52,951 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:52,952 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264122952 2023-02-13 04:54:52,952 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:52,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:52,955 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:52,955 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:52,956 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:52,956 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264152956 2023-02-13 04:54:52,956 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264212956 2023-02-13 04:54:52,956 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:52,956 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:52,959 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN] 2023-02-13 04:54:52,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:52,959 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:52,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:52,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:53,206 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,206 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:53,206 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:53,206 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:53,207 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:53,207 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:53,207 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:53,207 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:53,208 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:53,208 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:53,209 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 04:54:53,209 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:54:53,209 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:54:53,209 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 04:54:53,209 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:53,210 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:53,210 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:53,210 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:53,210 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,210 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:53,210 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,211 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,211 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,211 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:53,211 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,211 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,211 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:53,212 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,212 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:53,213 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:53,213 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:53,213 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:53,213 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,213 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,213 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,213 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:53,213 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:53,213 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:53,213 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:53,214 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:53,214 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:53,214 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:53,215 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264123215 2023-02-13 04:54:53,215 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:53,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,217 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:53,219 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:53,219 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:53,219 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:53,219 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264153219 2023-02-13 04:54:53,219 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264213219 2023-02-13 04:54:53,219 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:53,220 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:53,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:53,224 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:53,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:53,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:53,470 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:53,470 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:53,471 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:53,472 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:54:53,472 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 04:54:53,472 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:53,472 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:53,472 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:53,473 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:53,473 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,473 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:53,473 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,473 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,474 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,474 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:53,474 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,474 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,474 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:53,474 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,475 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:53,475 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:53,475 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,475 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,475 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,475 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:53,475 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-13 04:54:53,476 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:53,476 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:53,476 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:53,476 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264123476 2023-02-13 04:54:53,478 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:53,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,479 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:53,481 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:53,481 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:53,481 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:53,481 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264153481 2023-02-13 04:54:53,482 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264213481 2023-02-13 04:54:53,482 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:53,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:53,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:53,484 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:53,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:53,484 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:53,732 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,732 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:53,732 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:53,732 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:53,734 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:53,734 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:53,735 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:53,735 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:53,735 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:53,735 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:53,736 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 04:54:53,736 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 04:54:53,736 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:53,737 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:53,737 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:53,737 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:53,737 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,737 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:53,738 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,738 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:53,738 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,738 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:53,738 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,738 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:53,739 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:53,739 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,739 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:53,739 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:53,739 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,739 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,739 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,739 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:53,740 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:53,740 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:53,741 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:53,741 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:53,741 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264123741 2023-02-13 04:54:53,742 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:53,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,743 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:53,745 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:53,745 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:53,746 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:53,746 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264153746 2023-02-13 04:54:53,746 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264213746 2023-02-13 04:54:53,746 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 04:54:53,746 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:53,748 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:53,748 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 04:54:53,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:53,748 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:53,996 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:53,996 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:53,996 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:53,996 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:53,996 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:53,997 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:53,997 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:53,997 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:53,997 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:53,997 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:53,998 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 04:54:53,998 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 04:54:53,998 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:53,999 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:53,999 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:53,999 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:53,999 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:54,000 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:54,000 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:54,000 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:54,000 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:54,000 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:54,001 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:54,001 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:54,001 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,002 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:54,002 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,002 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:54,002 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:54,002 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:54,002 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:54,002 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:54:54,002 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 04:54:54,003 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:54,003 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:54,004 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:54,005 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264124005 2023-02-13 04:54:54,005 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:54,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:54,010 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,012 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:54,012 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:54,012 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:54,012 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264154012 2023-02-13 04:54:54,013 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264214013 2023-02-13 04:54:54,013 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 7 msec 2023-02-13 04:54:54,013 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:54:54,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 exec-time=4.4950 sec 2023-02-13 04:54:54,015 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,016 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=true 2023-02-13 04:54:54,166 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:54,167 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:54,167 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264094167"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264094167"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264094167"}]},"ts":"1676264094167"} 2023-02-13 04:54:54,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:54,169 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:54:54,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:54:54,170 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:54:54,263 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:54,263 WARN [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 04:54:54,263 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:54:54,263 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:54:54,263 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:54:54,263 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:54:54,263 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:54:54,263 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:54:54,264 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:54:54,264 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:54:54,265 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:54:54,266 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 04:54:54,266 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:54:54,266 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:54:54,267 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:54:54,267 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:54:54,267 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:54,267 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:54:54,268 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:54,268 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:54:54,268 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:54,269 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:54:54,269 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:54,269 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:54:54,269 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,270 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:54:54,270 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,270 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:54:54,270 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,270 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 04:54:54,271 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:54:54,271 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:54:54,272 INFO [Listener at localhost/33551] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6 to restore RIT 2023-02-13 04:54:54,274 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264124274 2023-02-13 04:54:54,275 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:54:54,275 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:54:54,277 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:54:54,278 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,46601,1676264072787, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787, openSeqNum=2 2023-02-13 04:54:54,278 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:54:54,278 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264154278 2023-02-13 04:54:54,278 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264214278 2023-02-13 04:54:54,278 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:54:54,278 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:54,279 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264094278"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264094278"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264094278"}]},"ts":"1676264094278"} 2023-02-13 04:54:54,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:54,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3bcdfe2dbd4e99a75b6c62f3d21619a6, NAME => 'testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:54,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:54,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:54,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,438 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,439 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 2023-02-13 04:54:54,439 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 2023-02-13 04:54:54,439 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bcdfe2dbd4e99a75b6c62f3d21619a6 columnFamilyName cf1 2023-02-13 04:54:54,446 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 2023-02-13 04:54:54,449 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 2023-02-13 04:54:54,451 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 2023-02-13 04:54:54,454 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 2023-02-13 04:54:54,456 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 2023-02-13 04:54:54,459 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf 2023-02-13 04:54:54,462 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 2023-02-13 04:54:54,462 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(310): Store=3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:54,462 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,463 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 2023-02-13 04:54:54,463 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 2023-02-13 04:54:54,463 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bcdfe2dbd4e99a75b6c62f3d21619a6 columnFamilyName cf2 2023-02-13 04:54:54,469 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 2023-02-13 04:54:54,472 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 2023-02-13 04:54:54,474 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 2023-02-13 04:54:54,477 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 2023-02-13 04:54:54,481 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 2023-02-13 04:54:54,484 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e 2023-02-13 04:54:54,486 DEBUG [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab 2023-02-13 04:54:54,487 INFO [StoreOpener-3bcdfe2dbd4e99a75b6c62f3d21619a6-1] regionserver.HStore(310): Store=3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:54,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.splits 2023-02-13 04:54:54,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:54,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3bcdfe2dbd4e99a75b6c62f3d21619a6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67539399, jitterRate=0.006415471434593201}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:54,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: Running coprocessor pre-open hook at 1676264094437Writing region info on filesystem at 1676264094437Initializing all the Stores at 1676264094438 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264094438Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264094438Cleaning up temporary data from old regions at 1676264094489 (+51 ms)Cleaning up detritus from prior splits at 1676264094489Running coprocessor post-open hooks at 1676264094492 (+3 ms)Region opened successfully at 1676264094492 2023-02-13 04:54:54,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6., pid=48, masterSystemTime=1676264094433 2023-02-13 04:54:54,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:54,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:54,494 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:54,494 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:54,495 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 04:54:54,495 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 04:54:54,495 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1912): 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:54,495 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1912): 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:54,495 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 in testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,495 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 in testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,496 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp, totalSize=34.9 K 2023-02-13 04:54:54,496 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp, totalSize=34.9 K 2023-02-13 04:54:54,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,497 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 51e47f2e2793464cb165b9a079cb5151, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264088110 2023-02-13 04:54:54,497 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:54,497 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting c50debe93c124539a0b6b1eb31da10cf, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264088110 2023-02-13 04:54:54,497 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264094497"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264094497"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264094497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264094497"}]},"ts":"1676264094497"} 2023-02-13 04:54:54,497 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting a6c2661ae8944403af4d99c86e942fd1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264088295 2023-02-13 04:54:54,497 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting aa6cd264297142efa2249c453e57aa43, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264088295 2023-02-13 04:54:54,497 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 7988f443f32b437f8880beca3ee71ab8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264088494 2023-02-13 04:54:54,498 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 42209a1628054e68adcfb5ce2018e888, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264088494 2023-02-13 04:54:54,498 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting eef0dc8cd3ae486d92de9a504660c7ab, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264088690 2023-02-13 04:54:54,498 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 6701697f5dfa431ca736d6551776a361, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264088690 2023-02-13 04:54:54,498 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting aff02efde588496ba6217076c9a5d8b7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264088877 2023-02-13 04:54:54,498 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting effc55ad0971459ab3f9801ba067e3e0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264088877 2023-02-13 04:54:54,499 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting c9a55339a3fb4abdab2ffb719068620e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264089076 2023-02-13 04:54:54,499 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 001cab63e8834d61b300afb089f0b200, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264089076 2023-02-13 04:54:54,499 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] compactions.Compactor(207): Compacting 3bdbf0b48c2844f996af19c084bc6f00, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264089258 2023-02-13 04:54:54,499 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] compactions.Compactor(207): Compacting 00d8e7b54cf143a0a5f6a4741947c274, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264089258 2023-02-13 04:54:54,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 04:54:54,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 in 218 msec 2023-02-13 04:54:54,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN in 1.5430 sec 2023-02-13 04:54:54,524 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3bcdfe2dbd4e99a75b6c62f3d21619a6#cf1#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:54,528 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] throttle.PressureAwareThroughputController(145): 3bcdfe2dbd4e99a75b6c62f3d21619a6#cf2#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:54,529 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 04:54:54,532 DEBUG [Listener at localhost/33551] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 04:54:54,534 INFO [Listener at localhost/33551] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 04:54:54,540 INFO [Listener at localhost/33551] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 04:54:54,541 INFO [Listener at localhost/33551] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 04:54:54,541 INFO [Listener at localhost/33551] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 04:54:54,554 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf1/30842601a78e4a089c966a8f5c1c5345 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/30842601a78e4a089c966a8f5c1c5345 2023-02-13 04:54:54,561 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/.tmp/cf2/a627cb7c2f824013876c72eaff9a71a3 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a627cb7c2f824013876c72eaff9a71a3 2023-02-13 04:54:54,563 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1 of 3bcdfe2dbd4e99a75b6c62f3d21619a6 into 30842601a78e4a089c966a8f5c1c5345(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:54,563 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:54,563 INFO [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6., storeName=3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1, priority=9, startTime=1676264094493; duration=0sec 2023-02-13 04:54:54,564 DEBUG [RS:0;jenkins-hbase4:46601-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:54,570 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2 of 3bcdfe2dbd4e99a75b6c62f3d21619a6 into a627cb7c2f824013876c72eaff9a71a3(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:54,570 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: 2023-02-13 04:54:54,570 INFO [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6., storeName=3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2, priority=9, startTime=1676264094493; duration=0sec 2023-02-13 04:54:54,571 DEBUG [RS:0;jenkins-hbase4:46601-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:54,580 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:54,582 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 04:54:54,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-13 04:54:54,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 04:54:54,586 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264094586"}]},"ts":"1676264094586"} 2023-02-13 04:54:54,587 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 04:54:54,591 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 04:54:54,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN}] 2023-02-13 04:54:54,593 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN 2023-02-13 04:54:54,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:54,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264094594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264094594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264094594"}]},"ts":"1676264094594"} 2023-02-13 04:54:54,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:54:54,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 04:54:54,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3bcdfe2dbd4e99a75b6c62f3d21619a6, disabling compactions & flushes 2023-02-13 04:54:54,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. after waiting 0 ms 2023-02-13 04:54:54,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,749 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274] to archive 2023-02-13 04:54:54,750 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:54,752 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/c50debe93c124539a0b6b1eb31da10cf 2023-02-13 04:54:54,753 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/aa6cd264297142efa2249c453e57aa43 2023-02-13 04:54:54,754 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/42209a1628054e68adcfb5ce2018e888 2023-02-13 04:54:54,754 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/6701697f5dfa431ca736d6551776a361 2023-02-13 04:54:54,756 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/effc55ad0971459ab3f9801ba067e3e0 2023-02-13 04:54:54,757 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/001cab63e8834d61b300afb089f0b200 2023-02-13 04:54:54,758 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/00d8e7b54cf143a0a5f6a4741947c274 2023-02-13 04:54:54,762 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00] to archive 2023-02-13 04:54:54,762 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:54:54,764 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/51e47f2e2793464cb165b9a079cb5151 2023-02-13 04:54:54,765 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a6c2661ae8944403af4d99c86e942fd1 2023-02-13 04:54:54,765 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/7988f443f32b437f8880beca3ee71ab8 2023-02-13 04:54:54,766 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/eef0dc8cd3ae486d92de9a504660c7ab 2023-02-13 04:54:54,767 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/aff02efde588496ba6217076c9a5d8b7 2023-02-13 04:54:54,768 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/c9a55339a3fb4abdab2ffb719068620e 2023-02-13 04:54:54,769 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/3bdbf0b48c2844f996af19c084bc6f00 2023-02-13 04:54:54,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:54:54,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:54,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6. 2023-02-13 04:54:54,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3bcdfe2dbd4e99a75b6c62f3d21619a6: Waiting for close lock at 1676264094748Running coprocessor pre-close hooks at 1676264094748Disabling compacts and flushes for region at 1676264094748Disabling writes for close at 1676264094748Writing region close event to WAL at 1676264094770 (+22 ms)Running coprocessor post-close hooks at 1676264094774 (+4 ms)Closed at 1676264094774 2023-02-13 04:54:54,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,776 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=3bcdfe2dbd4e99a75b6c62f3d21619a6, regionState=CLOSED 2023-02-13 04:54:54,776 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264094776"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264094776"}]},"ts":"1676264094776"} 2023-02-13 04:54:54,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 04:54:54,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 3bcdfe2dbd4e99a75b6c62f3d21619a6, server=jenkins-hbase4.apache.org,46601,1676264072787 in 181 msec 2023-02-13 04:54:54,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 04:54:54,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, UNASSIGN in 187 msec 2023-02-13 04:54:54,781 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264094781"}]},"ts":"1676264094781"} 2023-02-13 04:54:54,782 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 04:54:54,783 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 04:54:54,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 202 msec 2023-02-13 04:54:54,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 04:54:54,888 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 04:54:54,889 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-13 04:54:54,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 04:54:54,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,896 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,897 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/recovered.edits] 2023-02-13 04:54:54,900 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/30842601a78e4a089c966a8f5c1c5345 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf1/30842601a78e4a089c966a8f5c1c5345 2023-02-13 04:54:54,902 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a627cb7c2f824013876c72eaff9a71a3 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/cf2/a627cb7c2f824013876c72eaff9a71a3 2023-02-13 04:54:54,905 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6/recovered.edits/90.seqid 2023-02-13 04:54:54,905 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackAndDoubleExecution/3bcdfe2dbd4e99a75b6c62f3d21619a6 2023-02-13 04:54:54,906 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 04:54:54,909 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,911 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 04:54:54,913 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 04:54:54,914 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,914 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 04:54:54,914 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264094914"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:54,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 04:54:54,915 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3bcdfe2dbd4e99a75b6c62f3d21619a6, NAME => 'testRollbackAndDoubleExecution,,1676264087605.3bcdfe2dbd4e99a75b6c62f3d21619a6.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 04:54:54,915 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 04:54:54,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264094916"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:54,917 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 04:54:54,919 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:54:54,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-02-13 04:54:54,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 04:54:54,994 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 04:54:55,008 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=398 (was 401), OpenFileDescriptor=859 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 249), ProcessCount=168 (was 168), AvailableMemoryMB=3736 (was 3757) 2023-02-13 04:54:55,022 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=398, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=237, ProcessCount=168, AvailableMemoryMB=3735 2023-02-13 04:54:55,022 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:55,025 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:54:55,025 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:54:55,026 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:55,028 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:54:55,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:54:55,030 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,031 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 empty. 2023-02-13 04:54:55,031 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,031 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 04:54:55,042 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:55,043 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => ec38915ce1c4d08b104c37dce39eedc0, NAME => 'testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:54:55,050 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:55,050 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing ec38915ce1c4d08b104c37dce39eedc0, disabling compactions & flushes 2023-02-13 04:54:55,050 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,050 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,050 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. after waiting 0 ms 2023-02-13 04:54:55,050 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,050 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,050 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for ec38915ce1c4d08b104c37dce39eedc0: Waiting for close lock at 1676264095050Disabling compacts and flushes for region at 1676264095050Disabling writes for close at 1676264095050Writing region close event to WAL at 1676264095050Closed at 1676264095050 2023-02-13 04:54:55,052 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:54:55,053 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676264095053"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264095053"}]},"ts":"1676264095053"} 2023-02-13 04:54:55,055 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:54:55,056 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:54:55,056 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264095056"}]},"ts":"1676264095056"} 2023-02-13 04:54:55,057 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 04:54:55,060 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:55,061 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:55,061 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:55,061 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:55,061 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:55,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, ASSIGN}] 2023-02-13 04:54:55,063 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, ASSIGN 2023-02-13 04:54:55,064 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:55,214 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:55,215 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=ec38915ce1c4d08b104c37dce39eedc0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:55,215 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676264095215"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264095215"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264095215"}]},"ts":"1676264095215"} 2023-02-13 04:54:55,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure ec38915ce1c4d08b104c37dce39eedc0, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:55,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ec38915ce1c4d08b104c37dce39eedc0, NAME => 'testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:55,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:55,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:55,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,377 INFO [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,378 DEBUG [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/cf1 2023-02-13 04:54:55,378 DEBUG [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/cf1 2023-02-13 04:54:55,379 INFO [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec38915ce1c4d08b104c37dce39eedc0 columnFamilyName cf1 2023-02-13 04:54:55,379 INFO [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] regionserver.HStore(310): Store=ec38915ce1c4d08b104c37dce39eedc0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:55,379 INFO [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,381 DEBUG [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/cf2 2023-02-13 04:54:55,381 DEBUG [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/cf2 2023-02-13 04:54:55,381 INFO [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec38915ce1c4d08b104c37dce39eedc0 columnFamilyName cf2 2023-02-13 04:54:55,382 INFO [StoreOpener-ec38915ce1c4d08b104c37dce39eedc0-1] regionserver.HStore(310): Store=ec38915ce1c4d08b104c37dce39eedc0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:55,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:55,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:55,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ec38915ce1c4d08b104c37dce39eedc0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64612235, jitterRate=-0.03720267117023468}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:55,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ec38915ce1c4d08b104c37dce39eedc0: Running coprocessor pre-open hook at 1676264095376Writing region info on filesystem at 1676264095376Initializing all the Stores at 1676264095376Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264095376Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264095377 (+1 ms)Cleaning up temporary data from old regions at 1676264095384 (+7 ms)Cleaning up detritus from prior splits at 1676264095384Running coprocessor post-open hooks at 1676264095388 (+4 ms)Region opened successfully at 1676264095388 2023-02-13 04:54:55,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0., pid=55, masterSystemTime=1676264095369 2023-02-13 04:54:55,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,391 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=ec38915ce1c4d08b104c37dce39eedc0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:55,391 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676264095391"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264095391"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264095391"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264095391"}]},"ts":"1676264095391"} 2023-02-13 04:54:55,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 04:54:55,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure ec38915ce1c4d08b104c37dce39eedc0, server=jenkins-hbase4.apache.org,36009,1676264072859 in 175 msec 2023-02-13 04:54:55,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 04:54:55,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, ASSIGN in 333 msec 2023-02-13 04:54:55,396 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:54:55,396 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264095396"}]},"ts":"1676264095396"} 2023-02-13 04:54:55,397 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 04:54:55,401 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:54:55,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 375 msec 2023-02-13 04:54:55,527 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:55,528 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:55,529 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:55,529 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:55,529 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:55,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, UNASSIGN}] 2023-02-13 04:54:55,532 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, UNASSIGN 2023-02-13 04:54:55,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=ec38915ce1c4d08b104c37dce39eedc0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:55,533 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676264095532"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264095532"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264095532"}]},"ts":"1676264095532"} 2023-02-13 04:54:55,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure ec38915ce1c4d08b104c37dce39eedc0, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:55,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ec38915ce1c4d08b104c37dce39eedc0, disabling compactions & flushes 2023-02-13 04:54:55,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. after waiting 0 ms 2023-02-13 04:54:55,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 04:54:55,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:55,692 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0. 2023-02-13 04:54:55,692 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ec38915ce1c4d08b104c37dce39eedc0: Waiting for close lock at 1676264095686Running coprocessor pre-close hooks at 1676264095686Disabling compacts and flushes for region at 1676264095686Disabling writes for close at 1676264095687 (+1 ms)Writing region close event to WAL at 1676264095687Running coprocessor post-close hooks at 1676264095691 (+4 ms)Closed at 1676264095692 (+1 ms) 2023-02-13 04:54:55,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:55,694 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=ec38915ce1c4d08b104c37dce39eedc0, regionState=CLOSED 2023-02-13 04:54:55,694 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676264095694"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264095694"}]},"ts":"1676264095694"} 2023-02-13 04:54:55,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 04:54:55,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure ec38915ce1c4d08b104c37dce39eedc0, server=jenkins-hbase4.apache.org,36009,1676264072859 in 161 msec 2023-02-13 04:54:55,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 04:54:55,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=ec38915ce1c4d08b104c37dce39eedc0, UNASSIGN in 165 msec 2023-02-13 04:54:55,721 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 04:54:55,723 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 04:54:55,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676264095724"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676264095724"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676264095724"}]},"ts":"1676264095724"} 2023-02-13 04:54:55,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264095724"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264095724"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264095724"}]},"ts":"1676264095724"} 2023-02-13 04:54:55,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264095724"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264095724"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264095724"}]},"ts":"1676264095724"} 2023-02-13 04:54:55,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, ASSIGN}] 2023-02-13 04:54:55,728 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, ASSIGN 2023-02-13 04:54:55,729 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:55,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, ASSIGN 2023-02-13 04:54:55,730 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:55,879 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:54:55,881 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=2dece162b7a5eb9a13add441b9de6ecb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:55,881 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264095881"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264095881"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264095881"}]},"ts":"1676264095881"} 2023-02-13 04:54:55,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 2dece162b7a5eb9a13add441b9de6ecb, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:55,885 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=3071776f1f3f38a1657d8b0a287d44a1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:55,885 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264095885"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264095885"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264095885"}]},"ts":"1676264095885"} 2023-02-13 04:54:55,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 3071776f1f3f38a1657d8b0a287d44a1, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:56,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3071776f1f3f38a1657d8b0a287d44a1, NAME => 'testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 04:54:56,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:56,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:56,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,039 INFO [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,040 DEBUG [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/cf1 2023-02-13 04:54:56,040 DEBUG [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/cf1 2023-02-13 04:54:56,041 INFO [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3071776f1f3f38a1657d8b0a287d44a1 columnFamilyName cf1 2023-02-13 04:54:56,041 INFO [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] regionserver.HStore(310): Store=3071776f1f3f38a1657d8b0a287d44a1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:56,041 INFO [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,042 DEBUG [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/cf2 2023-02-13 04:54:56,042 DEBUG [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/cf2 2023-02-13 04:54:56,043 INFO [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3071776f1f3f38a1657d8b0a287d44a1 columnFamilyName cf2 2023-02-13 04:54:56,043 INFO [StoreOpener-3071776f1f3f38a1657d8b0a287d44a1-1] regionserver.HStore(310): Store=3071776f1f3f38a1657d8b0a287d44a1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:56,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:56,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3071776f1f3f38a1657d8b0a287d44a1; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74858713, jitterRate=0.11548174917697906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:56,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3071776f1f3f38a1657d8b0a287d44a1: Running coprocessor pre-open hook at 1676264096038Writing region info on filesystem at 1676264096038Initializing all the Stores at 1676264096039 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264096039Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264096039Cleaning up temporary data from old regions at 1676264096045 (+6 ms)Cleaning up detritus from prior splits at 1676264096046 (+1 ms)Running coprocessor post-open hooks at 1676264096048 (+2 ms)Region opened successfully at 1676264096048 2023-02-13 04:54:56,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1., pid=62, masterSystemTime=1676264096034 2023-02-13 04:54:56,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2dece162b7a5eb9a13add441b9de6ecb, NAME => 'testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 04:54:56,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=3071776f1f3f38a1657d8b0a287d44a1, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:56,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:56,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,051 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264096050"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264096050"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264096050"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264096050"}]},"ts":"1676264096050"} 2023-02-13 04:54:56,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:56,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,052 INFO [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,053 DEBUG [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/cf1 2023-02-13 04:54:56,053 DEBUG [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/cf1 2023-02-13 04:54:56,054 INFO [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2dece162b7a5eb9a13add441b9de6ecb columnFamilyName cf1 2023-02-13 04:54:56,054 INFO [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] regionserver.HStore(310): Store=2dece162b7a5eb9a13add441b9de6ecb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:56,054 INFO [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,055 DEBUG [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/cf2 2023-02-13 04:54:56,055 DEBUG [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/cf2 2023-02-13 04:54:56,056 INFO [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2dece162b7a5eb9a13add441b9de6ecb columnFamilyName cf2 2023-02-13 04:54:56,056 INFO [StoreOpener-2dece162b7a5eb9a13add441b9de6ecb-1] regionserver.HStore(310): Store=2dece162b7a5eb9a13add441b9de6ecb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:56,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-13 04:54:56,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 3071776f1f3f38a1657d8b0a287d44a1, server=jenkins-hbase4.apache.org,36009,1676264072859 in 166 msec 2023-02-13 04:54:56,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, ASSIGN in 331 msec 2023-02-13 04:54:56,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:56,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2dece162b7a5eb9a13add441b9de6ecb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60305616, jitterRate=-0.10137629508972168}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:56,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2dece162b7a5eb9a13add441b9de6ecb: Running coprocessor pre-open hook at 1676264096051Writing region info on filesystem at 1676264096051Initializing all the Stores at 1676264096052 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264096052Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264096052Cleaning up temporary data from old regions at 1676264096059 (+7 ms)Cleaning up detritus from prior splits at 1676264096059Running coprocessor post-open hooks at 1676264096062 (+3 ms)Region opened successfully at 1676264096062 2023-02-13 04:54:56,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb., pid=61, masterSystemTime=1676264096034 2023-02-13 04:54:56,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=2dece162b7a5eb9a13add441b9de6ecb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:56,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264096064"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264096064"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264096064"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264096064"}]},"ts":"1676264096064"} 2023-02-13 04:54:56,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-13 04:54:56,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 2dece162b7a5eb9a13add441b9de6ecb, server=jenkins-hbase4.apache.org,36009,1676264072859 in 183 msec 2023-02-13 04:54:56,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-13 04:54:56,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, ASSIGN in 341 msec 2023-02-13 04:54:56,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 in 541 msec 2023-02-13 04:54:56,282 DEBUG [hconnection-0x4d838b63-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 04:54:56,286 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45704, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 04:54:56,288 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:54:56,291 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:56,292 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,293 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 04:54:56,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264096296"}]},"ts":"1676264096296"} 2023-02-13 04:54:56,297 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 04:54:56,299 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 04:54:56,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, UNASSIGN}] 2023-02-13 04:54:56,300 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, UNASSIGN 2023-02-13 04:54:56,301 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=3071776f1f3f38a1657d8b0a287d44a1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:56,301 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264096301"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264096301"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264096301"}]},"ts":"1676264096301"} 2023-02-13 04:54:56,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 3071776f1f3f38a1657d8b0a287d44a1, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:56,305 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, UNASSIGN 2023-02-13 04:54:56,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=2dece162b7a5eb9a13add441b9de6ecb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:56,306 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264096305"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264096305"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264096305"}]},"ts":"1676264096305"} 2023-02-13 04:54:56,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 2dece162b7a5eb9a13add441b9de6ecb, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:56,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 04:54:56,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3071776f1f3f38a1657d8b0a287d44a1, disabling compactions & flushes 2023-02-13 04:54:56,455 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. after waiting 0 ms 2023-02-13 04:54:56,455 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 04:54:56,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:56,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1. 2023-02-13 04:54:56,460 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3071776f1f3f38a1657d8b0a287d44a1: Waiting for close lock at 1676264096454Running coprocessor pre-close hooks at 1676264096454Disabling compacts and flushes for region at 1676264096454Disabling writes for close at 1676264096455 (+1 ms)Writing region close event to WAL at 1676264096456 (+1 ms)Running coprocessor post-close hooks at 1676264096459 (+3 ms)Closed at 1676264096460 (+1 ms) 2023-02-13 04:54:56,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2dece162b7a5eb9a13add441b9de6ecb, disabling compactions & flushes 2023-02-13 04:54:56,462 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. after waiting 0 ms 2023-02-13 04:54:56,462 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=3071776f1f3f38a1657d8b0a287d44a1, regionState=CLOSED 2023-02-13 04:54:56,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264096462"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264096462"}]},"ts":"1676264096462"} 2023-02-13 04:54:56,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 04:54:56,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 04:54:56,466 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 3071776f1f3f38a1657d8b0a287d44a1, server=jenkins-hbase4.apache.org,36009,1676264072859 in 162 msec 2023-02-13 04:54:56,466 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:56,467 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb. 2023-02-13 04:54:56,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2dece162b7a5eb9a13add441b9de6ecb: Waiting for close lock at 1676264096461Running coprocessor pre-close hooks at 1676264096461Disabling compacts and flushes for region at 1676264096461Disabling writes for close at 1676264096462 (+1 ms)Writing region close event to WAL at 1676264096462Running coprocessor post-close hooks at 1676264096466 (+4 ms)Closed at 1676264096467 (+1 ms) 2023-02-13 04:54:56,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3071776f1f3f38a1657d8b0a287d44a1, UNASSIGN in 167 msec 2023-02-13 04:54:56,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,468 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=2dece162b7a5eb9a13add441b9de6ecb, regionState=CLOSED 2023-02-13 04:54:56,468 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264096468"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264096468"}]},"ts":"1676264096468"} 2023-02-13 04:54:56,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 04:54:56,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 2dece162b7a5eb9a13add441b9de6ecb, server=jenkins-hbase4.apache.org,36009,1676264072859 in 162 msec 2023-02-13 04:54:56,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-13 04:54:56,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2dece162b7a5eb9a13add441b9de6ecb, UNASSIGN in 172 msec 2023-02-13 04:54:56,473 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264096472"}]},"ts":"1676264096472"} 2023-02-13 04:54:56,474 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 04:54:56,476 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 04:54:56,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 183 msec 2023-02-13 04:54:56,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 04:54:56,598 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 04:54:56,599 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,601 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,602 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 04:54:56,607 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:56,607 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,607 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,609 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/recovered.edits] 2023-02-13 04:54:56,610 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/recovered.edits] 2023-02-13 04:54:56,610 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/recovered.edits] 2023-02-13 04:54:56,617 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/recovered.edits/7.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1/recovered.edits/7.seqid 2023-02-13 04:54:56,617 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/recovered.edits/7.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb/recovered.edits/7.seqid 2023-02-13 04:54:56,617 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/recovered.edits/4.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0/recovered.edits/4.seqid 2023-02-13 04:54:56,617 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:54:56,618 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/2dece162b7a5eb9a13add441b9de6ecb 2023-02-13 04:54:56,618 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionNoStoreFile/ec38915ce1c4d08b104c37dce39eedc0 2023-02-13 04:54:56,618 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 04:54:56,620 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,622 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 04:54:56,623 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 04:54:56,624 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,624 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 04:54:56,624 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264096624"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:56,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264096624"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:56,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264096624"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:56,626 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:54:56,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ec38915ce1c4d08b104c37dce39eedc0, NAME => 'testSplitTableRegionNoStoreFile,,1676264095026.ec38915ce1c4d08b104c37dce39eedc0.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2dece162b7a5eb9a13add441b9de6ecb, NAME => 'testSplitTableRegionNoStoreFile,,1676264095528.2dece162b7a5eb9a13add441b9de6ecb.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 3071776f1f3f38a1657d8b0a287d44a1, NAME => 'testSplitTableRegionNoStoreFile,41,1676264095528.3071776f1f3f38a1657d8b0a287d44a1.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 04:54:56,626 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 04:54:56,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264096626"}]},"ts":"9223372036854775807"} 2023-02-13 04:54:56,627 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 04:54:56,630 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:54:56,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 31 msec 2023-02-13 04:54:56,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 04:54:56,703 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 04:54:56,721 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406 (was 398) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-127175606_15 at /127.0.0.1:35894 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-127175606_15 at /127.0.0.1:55632 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 237), ProcessCount=168 (was 168), AvailableMemoryMB=3733 (was 3735) 2023-02-13 04:54:56,740 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=406, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=237, ProcessCount=168, AvailableMemoryMB=3733 2023-02-13 04:54:56,740 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:54:56,743 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:54:56,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:54:56,746 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:56,748 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:54:56,749 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:54:56,751 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:56,752 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b empty. 2023-02-13 04:54:56,752 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:56,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 04:54:56,784 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 04:54:56,785 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => e4e717d13d69df81025587937cd5f72b, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:54:56,794 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:56,794 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing e4e717d13d69df81025587937cd5f72b, disabling compactions & flushes 2023-02-13 04:54:56,794 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:56,794 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:56,794 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. after waiting 0 ms 2023-02-13 04:54:56,794 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:56,794 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:56,794 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for e4e717d13d69df81025587937cd5f72b: Waiting for close lock at 1676264096794Disabling compacts and flushes for region at 1676264096794Disabling writes for close at 1676264096794Writing region close event to WAL at 1676264096794Closed at 1676264096794 2023-02-13 04:54:56,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:54:56,797 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676264096797"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264096797"}]},"ts":"1676264096797"} 2023-02-13 04:54:56,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:54:56,799 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:54:56,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264096799"}]},"ts":"1676264096799"} 2023-02-13 04:54:56,800 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 04:54:56,804 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:54:56,805 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:54:56,805 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:54:56,805 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:54:56,805 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:54:56,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, ASSIGN}] 2023-02-13 04:54:56,807 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, ASSIGN 2023-02-13 04:54:56,808 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:56,958 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:54:56,959 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e4e717d13d69df81025587937cd5f72b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:56,959 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676264096959"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264096959"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264096959"}]},"ts":"1676264096959"} 2023-02-13 04:54:56,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure e4e717d13d69df81025587937cd5f72b, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:57,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e4e717d13d69df81025587937cd5f72b, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:54:57,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:57,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:57,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,118 INFO [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,119 DEBUG [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1 2023-02-13 04:54:57,119 DEBUG [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1 2023-02-13 04:54:57,119 INFO [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e4e717d13d69df81025587937cd5f72b columnFamilyName cf1 2023-02-13 04:54:57,120 INFO [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] regionserver.HStore(310): Store=e4e717d13d69df81025587937cd5f72b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:57,120 INFO [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,121 DEBUG [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2 2023-02-13 04:54:57,121 DEBUG [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2 2023-02-13 04:54:57,121 INFO [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e4e717d13d69df81025587937cd5f72b columnFamilyName cf2 2023-02-13 04:54:57,122 INFO [StoreOpener-e4e717d13d69df81025587937cd5f72b-1] regionserver.HStore(310): Store=e4e717d13d69df81025587937cd5f72b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:57,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:57,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:57,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:54:57,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e4e717d13d69df81025587937cd5f72b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58994340, jitterRate=-0.12091583013534546}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:57,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e4e717d13d69df81025587937cd5f72b: Running coprocessor pre-open hook at 1676264097117Writing region info on filesystem at 1676264097117Initializing all the Stores at 1676264097117Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264097117Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264097117Cleaning up temporary data from old regions at 1676264097124 (+7 ms)Cleaning up detritus from prior splits at 1676264097124Running coprocessor post-open hooks at 1676264097129 (+5 ms)Region opened successfully at 1676264097129 2023-02-13 04:54:57,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b., pid=71, masterSystemTime=1676264097112 2023-02-13 04:54:57,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,132 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e4e717d13d69df81025587937cd5f72b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,132 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676264097132"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264097132"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264097132"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264097132"}]},"ts":"1676264097132"} 2023-02-13 04:54:57,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 04:54:57,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure e4e717d13d69df81025587937cd5f72b, server=jenkins-hbase4.apache.org,36009,1676264072859 in 173 msec 2023-02-13 04:54:57,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 04:54:57,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, ASSIGN in 329 msec 2023-02-13 04:54:57,137 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:54:57,137 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264097137"}]},"ts":"1676264097137"} 2023-02-13 04:54:57,138 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 04:54:57,141 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:54:57,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 396 msec 2023-02-13 04:54:57,254 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:57,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,255 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,255 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,255 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:57,255 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:57,256 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,256 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,258 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,258 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,258 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,258 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,259 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,259 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,260 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,260 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:57,260 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:57,260 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,260 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:57,261 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:54:57,262 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,262 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,262 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,262 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,262 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,262 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,279 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 2023-02-13 04:54:57,299 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/0a74f0d9cebc43189011aa2d19dda0de 2023-02-13 04:54:57,305 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 2023-02-13 04:54:57,312 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:57,313 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/0a74f0d9cebc43189011aa2d19dda0de as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de 2023-02-13 04:54:57,318 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:54:57,319 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 58ms, sequenceid=6, compaction requested=false 2023-02-13 04:54:57,319 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,320 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:57,320 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,320 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:57,320 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:57,320 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,320 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,320 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,322 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,322 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,322 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,322 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:57,324 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41ef4d44[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:57,324 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:57,324 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:57,326 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,326 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:57,326 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:57,328 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,328 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,328 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,329 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,329 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,329 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,329 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,330 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,330 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,330 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:57,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:57,333 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:57,333 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:57,333 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:57,333 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,333 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,335 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,336 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,337 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,337 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,338 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,339 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,339 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,346 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:57,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,345 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,345 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,346 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,446 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:57,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:57,460 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:57,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:57,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,462 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,462 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,462 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:57,462 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:57,463 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,463 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,464 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,464 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,464 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,464 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,465 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,465 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,466 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:57,467 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,467 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,467 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,469 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,476 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/317c03b3799c41bc974cc02d5513a41a 2023-02-13 04:54:57,487 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/c553694bbd224129a9d9dc0d99f88da9 2023-02-13 04:54:57,491 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/317c03b3799c41bc974cc02d5513a41a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a 2023-02-13 04:54:57,495 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:57,496 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/c553694bbd224129a9d9dc0d99f88da9 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9 2023-02-13 04:54:57,499 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:54:57,500 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 33ms, sequenceid=19, compaction requested=false 2023-02-13 04:54:57,500 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:57,500 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,500 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:57,500 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:57,500 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,500 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,500 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,502 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,502 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,503 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4185ab9[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:57,504 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:57,504 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,505 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,505 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,506 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:57,505 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:57,506 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:57,507 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,507 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,507 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,508 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,508 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:57,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:57,511 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:57,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:57,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:57,511 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,512 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,515 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,516 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,516 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,518 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,518 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,518 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,521 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:57,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,522 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,622 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:57,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:57,636 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:57,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:57,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,637 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,638 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,638 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:57,638 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:57,638 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,638 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,639 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,639 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,640 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,640 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,641 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,641 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:57,642 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:57,642 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,642 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:57,642 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:57,643 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,645 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,645 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,645 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,645 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,645 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,651 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/585a3b149d7349ea8b28225213c9ac8e 2023-02-13 04:54:57,664 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 2023-02-13 04:54:57,668 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/585a3b149d7349ea8b28225213c9ac8e as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e 2023-02-13 04:54:57,672 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:57,673 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 2023-02-13 04:54:57,677 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:54:57,678 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 36ms, sequenceid=32, compaction requested=true 2023-02-13 04:54:57,678 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:57,678 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,678 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:57,678 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:57,678 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,679 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,679 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,683 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,683 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,683 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,683 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:57,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f9f04d9[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:57,685 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:57,685 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,686 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,687 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,687 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:57,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,687 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:57,688 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,688 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,688 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,690 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,690 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,690 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,690 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,690 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,690 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,690 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:57,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:57,693 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:57,693 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:57,693 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:57,693 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,693 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,696 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,695 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,696 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,696 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,698 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,699 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,699 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,701 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,701 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:57,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:57,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,702 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,703 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,703 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,802 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:57,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:57,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:57,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:57,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,821 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,821 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,821 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:57,821 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:57,822 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,822 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,824 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,824 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,824 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,824 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,825 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,825 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:57,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:57,828 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,829 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,829 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:57,829 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:57,829 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,829 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:57,829 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:57,830 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,830 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,830 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,830 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,830 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,830 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,847 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/458f4e2749f2498cb756c4bfe88b8a05 2023-02-13 04:54:57,863 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/d38c1b8fdb42486e948adc379d4a4a88 2023-02-13 04:54:57,869 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/458f4e2749f2498cb756c4bfe88b8a05 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05 2023-02-13 04:54:57,875 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:57,876 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/d38c1b8fdb42486e948adc379d4a4a88 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88 2023-02-13 04:54:57,882 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:54:57,883 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 54ms, sequenceid=45, compaction requested=true 2023-02-13 04:54:57,883 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:57,883 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:57,883 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:57,883 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:57,883 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,883 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:57,883 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:57,885 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,885 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,885 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,885 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:57,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:57,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cdcb729[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:57,887 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:57,887 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,889 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,889 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,889 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,890 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,890 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:57,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:57,890 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:57,892 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,892 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,892 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,892 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,892 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,892 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,893 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:57,893 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,893 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,893 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:57,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:57,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:57,897 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:57,897 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:57,897 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:57,897 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,897 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:57,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,905 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:57,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:57,906 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,906 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:57,907 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:57,909 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,911 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,911 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,915 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:57,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:57,915 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:57,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:57,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,914 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,915 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:57,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,915 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:57,915 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:57,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:57,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,016 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:58,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:58,030 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:58,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:58,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,032 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,032 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,032 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:58,032 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:58,032 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,032 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,034 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,034 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,034 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,034 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,035 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,035 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,036 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,036 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,036 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,036 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,037 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:58,039 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,039 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,040 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,040 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:58,050 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/a4a0c3435b5747cda1ab348f04a0028a 2023-02-13 04:54:58,067 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/664fe50ae4534c86a1dd8fc4e503cfab 2023-02-13 04:54:58,073 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/a4a0c3435b5747cda1ab348f04a0028a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a 2023-02-13 04:54:58,079 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:58,079 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/664fe50ae4534c86a1dd8fc4e503cfab as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab 2023-02-13 04:54:58,084 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:54:58,085 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 45ms, sequenceid=58, compaction requested=true 2023-02-13 04:54:58,086 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:58,086 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,089 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,089 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,089 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:58,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e2589b[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:58,091 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:58,091 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,093 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,093 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:58,094 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:58,094 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:58,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,096 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,097 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,097 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,097 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,097 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,097 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,097 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:58,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:58,101 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:58,101 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:58,101 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:58,101 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,101 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,103 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,104 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,105 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,105 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,108 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,109 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,110 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,113 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,114 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,114 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:58,114 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,114 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:58,115 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:58,115 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,115 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,115 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,215 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:58,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:58,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:58,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:58,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,236 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,236 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,236 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:58,236 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:58,237 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,237 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,241 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,241 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,241 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,241 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,242 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,242 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:58,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,244 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,244 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,244 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,244 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,248 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,248 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,248 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,248 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,248 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,248 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:58,248 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,249 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,249 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,249 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,249 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:54:58,250 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,251 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,251 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,251 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,251 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,251 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,262 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/7b68164361124734bb4f1e3b95c4588d 2023-02-13 04:54:58,280 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/6bf82868aa154b5e8269e13b7c3be6a0 2023-02-13 04:54:58,286 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/7b68164361124734bb4f1e3b95c4588d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d 2023-02-13 04:54:58,290 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:58,291 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/6bf82868aa154b5e8269e13b7c3be6a0 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0 2023-02-13 04:54:58,295 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:54:58,296 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 47ms, sequenceid=71, compaction requested=true 2023-02-13 04:54:58,296 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:58,296 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,296 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:58,296 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:58,296 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,296 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,296 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,298 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,299 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:58,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41e9174c[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:58,300 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:58,300 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,303 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:58,303 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:58,305 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,305 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,305 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,306 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,306 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,307 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:58,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:58,310 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:58,310 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:58,310 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:58,310 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,310 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,313 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,314 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,314 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,314 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,316 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,316 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,316 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,319 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,319 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,319 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:58,320 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,319 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,420 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:58,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:58,433 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:58,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:58,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,435 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,435 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,435 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:58,435 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:58,435 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,435 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,436 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,436 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,436 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,436 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,437 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,437 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:58,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,438 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,438 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,438 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,438 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:58,439 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:58,439 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,439 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:58,439 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 04:54:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,441 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,448 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/d17d8448721a406793070feb1407d7d7 2023-02-13 04:54:58,452 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17d8448721a406793070feb1407d7d7 2023-02-13 04:54:58,460 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:54:58,463 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:54:58,464 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/d17d8448721a406793070feb1407d7d7 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7 2023-02-13 04:54:58,468 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17d8448721a406793070feb1407d7d7 2023-02-13 04:54:58,468 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 04:54:58,468 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:54:58,472 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:54:58,472 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 04:54:58,473 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 34ms, sequenceid=83, compaction requested=true 2023-02-13 04:54:58,473 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:58,473 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,473 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:58,473 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:58,473 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,473 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,473 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,475 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,475 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,476 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,476 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:58,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66f76a64[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:58,478 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:58,478 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,479 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,479 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,479 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,480 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,480 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:58,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,480 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:58,483 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,485 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,485 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,485 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:58,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:58,488 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:58,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:58,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:58,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,488 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:58,489 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,489 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,489 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,489 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,489 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,489 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,490 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,490 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,490 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,491 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,492 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,492 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,493 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,495 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,495 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:58,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,496 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,596 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:58,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:58,605 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:58,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:58,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,606 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:58,606 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:58,607 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,607 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,608 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:58,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,610 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,610 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,610 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,610 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 04:54:58,612 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,613 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,613 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,613 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,613 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,613 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,619 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:58,623 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:58,630 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:58,634 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:58,634 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/418946b19cdb436cbcc8474c5d90478a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:58,638 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:58,638 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 04:54:58,639 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/668e49c4370a4f07881dbd5e0fe1cb05 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:58,642 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:58,642 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 04:54:58,643 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 33ms, sequenceid=91, compaction requested=true 2023-02-13 04:54:58,643 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:58,643 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,643 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:58,643 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:58,643 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,643 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,643 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,645 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,645 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,645 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,645 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:58,647 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f4bec6c[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:58,647 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:58,647 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,648 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,648 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,648 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,649 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:58,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,649 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,649 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,649 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,649 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:58,648 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:58,649 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,650 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,652 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,652 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,652 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,652 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:58,656 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:58,656 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:58,656 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:58,656 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:58,656 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,656 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:58,658 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,659 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,659 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,660 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,661 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,662 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,662 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,663 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,663 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,667 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,667 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,690 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,667 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,690 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,691 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-02-13 04:54:58,690 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,691 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,690 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,691 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,690 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,691 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:58,691 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:58,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,691 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,691 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,791 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:58,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:58,800 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:54:58,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:54:58,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,802 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,802 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,802 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 04:54:58,802 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 04:54:58,802 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,802 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,804 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,804 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,804 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,804 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,804 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,805 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:54:58,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:54:58,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:54:58,806 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,806 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:54:58,806 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. started... 2023-02-13 04:54:58,806 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 04:54:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,809 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,816 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/54727de44d994daeb2938f064e694275 2023-02-13 04:54:58,821 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54727de44d994daeb2938f064e694275 2023-02-13 04:54:58,829 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:58,832 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:58,832 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/54727de44d994daeb2938f064e694275 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275 2023-02-13 04:54:58,836 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54727de44d994daeb2938f064e694275 2023-02-13 04:54:58,836 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 04:54:58,837 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/bcd01a3af22e4c8982b3a894c2a53e1d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:58,840 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:58,840 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 04:54:58,841 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 35ms, sequenceid=99, compaction requested=true 2023-02-13 04:54:58,842 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4e717d13d69df81025587937cd5f72b: 2023-02-13 04:54:58,842 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:58,842 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:54:58,842 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:54:58,842 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,842 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 04:54:58,842 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 04:54:58,844 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,844 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,844 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,844 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:54:58,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 04:54:58,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2af84d9e[Count = 0] remaining members to acquire global barrier 2023-02-13 04:54:58,846 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 04:54:58,846 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:54:58,849 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,849 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:54:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 04:54:58,849 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:54:58,850 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,850 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 04:54:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,851 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 04:54:58,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:54:58,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:54:58,854 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 04:54:58,854 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:54:58,854 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:54:58,854 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,854 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:54:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,855 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:54:58,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,856 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:54:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,858 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:54:58,858 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,858 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:54:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:54:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 04:54:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:54:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:54:58,862 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:54:58,862 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,863 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:54:58,863 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:54:58,963 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:54:58,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:54:58,967 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,967 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:58,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:58,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:58,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:58,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, UNASSIGN}] 2023-02-13 04:54:58,971 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, UNASSIGN 2023-02-13 04:54:58,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e4e717d13d69df81025587937cd5f72b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:58,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676264098971"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264098971"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264098971"}]},"ts":"1676264098971"} 2023-02-13 04:54:58,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure e4e717d13d69df81025587937cd5f72b, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:59,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e4e717d13d69df81025587937cd5f72b, disabling compactions & flushes 2023-02-13 04:54:59,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:59,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:59,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. after waiting 0 ms 2023-02-13 04:54:59,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:59,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e4e717d13d69df81025587937cd5f72b 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 04:54:59,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,150 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf1/2b50fd89a25446579ea95239b91e387a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 04:54:59,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.tmp/cf2/f59eb27353c44b59b49591170a4c5bfc as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 04:54:59,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 34ms, sequenceid=103, compaction requested=true 2023-02-13 04:54:59,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 04:54:59,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:54:59,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. 2023-02-13 04:54:59,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e4e717d13d69df81025587937cd5f72b: Waiting for close lock at 1676264099125Running coprocessor pre-close hooks at 1676264099125Disabling compacts and flushes for region at 1676264099125Disabling writes for close at 1676264099125Obtaining lock to block concurrent updates at 1676264099125Preparing flush snapshotting stores in e4e717d13d69df81025587937cd5f72b at 1676264099125Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676264099125Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b. at 1676264099126 (+1 ms)Flushing e4e717d13d69df81025587937cd5f72b/cf1: creating writer at 1676264099126Flushing e4e717d13d69df81025587937cd5f72b/cf1: appending metadata at 1676264099128 (+2 ms)Flushing e4e717d13d69df81025587937cd5f72b/cf1: closing flushed file at 1676264099128Flushing e4e717d13d69df81025587937cd5f72b/cf2: creating writer at 1676264099138 (+10 ms)Flushing e4e717d13d69df81025587937cd5f72b/cf2: appending metadata at 1676264099140 (+2 ms)Flushing e4e717d13d69df81025587937cd5f72b/cf2: closing flushed file at 1676264099140Flushing e4e717d13d69df81025587937cd5f72b/cf1: reopening flushed file at 1676264099151 (+11 ms)Flushing e4e717d13d69df81025587937cd5f72b/cf2: reopening flushed file at 1676264099155 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e4e717d13d69df81025587937cd5f72b in 34ms, sequenceid=103, compaction requested=true at 1676264099160 (+5 ms)Writing region close event to WAL at 1676264099161 (+1 ms)Running coprocessor post-close hooks at 1676264099164 (+3 ms)Closed at 1676264099164 2023-02-13 04:54:59,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,166 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e4e717d13d69df81025587937cd5f72b, regionState=CLOSED 2023-02-13 04:54:59,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676264099166"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264099166"}]},"ts":"1676264099166"} 2023-02-13 04:54:59,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 04:54:59,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure e4e717d13d69df81025587937cd5f72b, server=jenkins-hbase4.apache.org,36009,1676264072859 in 194 msec 2023-02-13 04:54:59,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 04:54:59,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e4e717d13d69df81025587937cd5f72b, UNASSIGN in 198 msec 2023-02-13 04:54:59,182 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=e4e717d13d69df81025587937cd5f72b, threads=16 2023-02-13 04:54:59,182 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,182 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,182 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,182 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,183 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,185 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,185 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,185 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,188 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,188 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,188 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,189 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,189 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,191 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,192 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,192 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,200 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:59,202 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,205 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,205 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a, top=true 2023-02-13 04:54:59,210 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,210 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88, top=false 2023-02-13 04:54:59,210 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0, top=false 2023-02-13 04:54:59,215 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:59,216 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:54:59,217 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05, top=false 2023-02-13 04:54:59,218 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:59,219 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,219 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc, top=true 2023-02-13 04:54:59,219 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab, top=false 2023-02-13 04:54:59,220 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54727de44d994daeb2938f064e694275 2023-02-13 04:54:59,221 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:59,221 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05, top=true 2023-02-13 04:54:59,222 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:59,222 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a, top=true 2023-02-13 04:54:59,235 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a for child: bf5f3b76dbba55abd9f0e0ed2d9ac96a, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,235 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,235 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,241 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:59,241 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d, top=true 2023-02-13 04:54:59,250 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05 for child: bf5f3b76dbba55abd9f0e0ed2d9ac96a, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,250 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,250 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,250 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0 for child: b457ad1e952dae854296990d02485fa8, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,251 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88 for child: b457ad1e952dae854296990d02485fa8, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,257 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab for child: b457ad1e952dae854296990d02485fa8, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,258 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc for child: bf5f3b76dbba55abd9f0e0ed2d9ac96a, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,258 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05 for child: b457ad1e952dae854296990d02485fa8, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,258 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,258 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,260 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,260 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,261 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,261 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d for child: bf5f3b76dbba55abd9f0e0ed2d9ac96a, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,261 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,261 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a for child: bf5f3b76dbba55abd9f0e0ed2d9ac96a, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,261 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,262 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d, top=false 2023-02-13 04:54:59,264 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54727de44d994daeb2938f064e694275 2023-02-13 04:54:59,264 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275, top=true 2023-02-13 04:54:59,267 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,272 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:54:59,272 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,273 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17d8448721a406793070feb1407d7d7 2023-02-13 04:54:59,274 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a, top=false 2023-02-13 04:54:59,295 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d for child: b457ad1e952dae854296990d02485fa8, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,312 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275 for child: bf5f3b76dbba55abd9f0e0ed2d9ac96a, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,312 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,318 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,327 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/.splits/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a for child: b457ad1e952dae854296990d02485fa8, parent: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,332 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,333 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,335 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,340 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17d8448721a406793070feb1407d7d7 2023-02-13 04:54:59,344 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,344 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,347 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,350 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,352 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,357 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 for region: e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,357 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region e4e717d13d69df81025587937cd5f72b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:54:59,402 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 04:54:59,404 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 04:54:59,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676264099405"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676264099405"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676264099405"}]},"ts":"1676264099405"} 2023-02-13 04:54:59,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264099405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264099405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264099405"}]},"ts":"1676264099405"} 2023-02-13 04:54:59,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264099405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264099405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264099405"}]},"ts":"1676264099405"} 2023-02-13 04:54:59,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, ASSIGN}] 2023-02-13 04:54:59,409 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, ASSIGN 2023-02-13 04:54:59,410 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:59,410 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, ASSIGN 2023-02-13 04:54:59,411 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:54:59,560 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:54:59,562 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=bf5f3b76dbba55abd9f0e0ed2d9ac96a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:59,562 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264099562"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264099562"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264099562"}]},"ts":"1676264099562"} 2023-02-13 04:54:59,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure bf5f3b76dbba55abd9f0e0ed2d9ac96a, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:59,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b457ad1e952dae854296990d02485fa8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:59,566 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264099565"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264099565"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264099565"}]},"ts":"1676264099565"} 2023-02-13 04:54:59,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure b457ad1e952dae854296990d02485fa8, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:54:59,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf5f3b76dbba55abd9f0e0ed2d9ac96a, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 04:54:59,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:59,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:59,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,719 INFO [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,720 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 2023-02-13 04:54:59,720 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 2023-02-13 04:54:59,720 INFO [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf5f3b76dbba55abd9f0e0ed2d9ac96a columnFamilyName cf1 2023-02-13 04:54:59,731 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7-top 2023-02-13 04:54:59,735 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a-top 2023-02-13 04:54:59,738 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e-top 2023-02-13 04:54:59,741 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,742 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7-top 2023-02-13 04:54:59,745 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,746 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a 2023-02-13 04:54:59,749 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:59,749 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:54:59,752 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 54727de44d994daeb2938f064e694275 2023-02-13 04:54:59,752 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275 2023-02-13 04:54:59,752 INFO [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(310): Store=bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:59,752 INFO [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,753 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 2023-02-13 04:54:59,753 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 2023-02-13 04:54:59,754 INFO [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf5f3b76dbba55abd9f0e0ed2d9ac96a columnFamilyName cf2 2023-02-13 04:54:59,764 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de-top 2023-02-13 04:54:59,767 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5-top 2023-02-13 04:54:59,771 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9-top 2023-02-13 04:54:59,774 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,774 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab-top 2023-02-13 04:54:59,777 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:59,777 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:54:59,779 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:59,779 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:54:59,782 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,782 DEBUG [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:54:59,783 INFO [StoreOpener-bf5f3b76dbba55abd9f0e0ed2d9ac96a-1] regionserver.HStore(310): Store=bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:59,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:59,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:54:59,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened bf5f3b76dbba55abd9f0e0ed2d9ac96a; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69344354, jitterRate=0.0333113968372345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:59,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for bf5f3b76dbba55abd9f0e0ed2d9ac96a: Running coprocessor pre-open hook at 1676264099718Writing region info on filesystem at 1676264099718Initializing all the Stores at 1676264099719 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264099719Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264099719Cleaning up temporary data from old regions at 1676264099785 (+66 ms)Cleaning up detritus from prior splits at 1676264099785Running coprocessor post-open hooks at 1676264099787 (+2 ms)Region opened successfully at 1676264099787 2023-02-13 04:54:59,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a., pid=77, masterSystemTime=1676264099715 2023-02-13 04:54:59,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:59,789 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:59,789 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:59,789 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,789 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1912): bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:59,789 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,789 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1912): bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:59,789 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,789 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,790 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp, totalSize=34.8 K 2023-02-13 04:54:59,790 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp, totalSize=34.8 K 2023-02-13 04:54:59,790 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264097249 2023-02-13 04:54:59,790 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting 0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264097249 2023-02-13 04:54:59,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,791 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264097448 2023-02-13 04:54:59,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,791 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264097448 2023-02-13 04:54:59,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b457ad1e952dae854296990d02485fa8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 04:54:59,791 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=bf5f3b76dbba55abd9f0e0ed2d9ac96a, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:59,791 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264097624 2023-02-13 04:54:59,791 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264099791"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264099791"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264099791"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264099791"}]},"ts":"1676264099791"} 2023-02-13 04:54:59,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:54:59,792 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676264098422 2023-02-13 04:54:59,791 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264097624 2023-02-13 04:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,792 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 04:54:59,792 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676264098422 2023-02-13 04:54:59,793 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 04:54:59,793 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 04:54:59,793 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 04:54:59,793 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 04:54:59,793 INFO [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,794 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 04:54:59,795 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1 2023-02-13 04:54:59,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-13 04:54:59,795 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1 2023-02-13 04:54:59,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure bf5f3b76dbba55abd9f0e0ed2d9ac96a, server=jenkins-hbase4.apache.org,36009,1676264072859 in 230 msec 2023-02-13 04:54:59,796 INFO [StoreOpener-b457ad1e952dae854296990d02485fa8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b457ad1e952dae854296990d02485fa8 columnFamilyName cf1 2023-02-13 04:54:59,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, ASSIGN in 387 msec 2023-02-13 04:54:59,809 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7-bottom 2023-02-13 04:54:59,812 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] throttle.PressureAwareThroughputController(145): bf5f3b76dbba55abd9f0e0ed2d9ac96a#cf2#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:59,812 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf5f3b76dbba55abd9f0e0ed2d9ac96a#cf1#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:59,814 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a-bottom 2023-02-13 04:54:59,817 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e-bottom 2023-02-13 04:54:59,820 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,820 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7-bottom 2023-02-13 04:54:59,827 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05 2023-02-13 04:54:59,834 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d 2023-02-13 04:54:59,837 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a 2023-02-13 04:54:59,839 INFO [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(310): Store=b457ad1e952dae854296990d02485fa8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:59,839 INFO [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,841 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2 2023-02-13 04:54:59,841 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2 2023-02-13 04:54:59,841 INFO [StoreOpener-b457ad1e952dae854296990d02485fa8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b457ad1e952dae854296990d02485fa8 columnFamilyName cf2 2023-02-13 04:54:59,849 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f911ceadc70a4eaab45aa42c87203c3e: NONE, but ROW specified in column family configuration 2023-02-13 04:54:59,850 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp/cf2/f911ceadc70a4eaab45aa42c87203c3e as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/f911ceadc70a4eaab45aa42c87203c3e 2023-02-13 04:54:59,852 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4b7819d725d94fb9ac0e101dc4b282d3: NONE, but ROW specified in column family configuration 2023-02-13 04:54:59,852 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de-bottom 2023-02-13 04:54:59,853 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp/cf1/4b7819d725d94fb9ac0e101dc4b282d3 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/4b7819d725d94fb9ac0e101dc4b282d3 2023-02-13 04:54:59,857 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5-bottom 2023-02-13 04:54:59,857 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for f911ceadc70a4eaab45aa42c87203c3e: NONE, but ROW specified in column family configuration 2023-02-13 04:54:59,858 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 of bf5f3b76dbba55abd9f0e0ed2d9ac96a into f911ceadc70a4eaab45aa42c87203c3e(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:59,858 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf5f3b76dbba55abd9f0e0ed2d9ac96a: 2023-02-13 04:54:59,858 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a., storeName=bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2, priority=9, startTime=1676264099788; duration=0sec 2023-02-13 04:54:59,859 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,860 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4b7819d725d94fb9ac0e101dc4b282d3: NONE, but ROW specified in column family configuration 2023-02-13 04:54:59,860 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9-bottom 2023-02-13 04:54:59,861 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 of bf5f3b76dbba55abd9f0e0ed2d9ac96a into 4b7819d725d94fb9ac0e101dc4b282d3(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:59,861 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf5f3b76dbba55abd9f0e0ed2d9ac96a: 2023-02-13 04:54:59,861 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a., storeName=bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1, priority=9, startTime=1676264099788; duration=0sec 2023-02-13 04:54:59,861 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,864 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:54:59,864 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab-bottom 2023-02-13 04:54:59,867 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab 2023-02-13 04:54:59,875 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0 2023-02-13 04:54:59,878 DEBUG [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88 2023-02-13 04:54:59,878 INFO [StoreOpener-b457ad1e952dae854296990d02485fa8-1] regionserver.HStore(310): Store=b457ad1e952dae854296990d02485fa8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:54:59,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:54:59,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b457ad1e952dae854296990d02485fa8 2023-02-13 04:54:59,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b457ad1e952dae854296990d02485fa8; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74456438, jitterRate=0.109487384557724}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:54:59,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b457ad1e952dae854296990d02485fa8: Running coprocessor pre-open hook at 1676264099792Writing region info on filesystem at 1676264099792Initializing all the Stores at 1676264099793 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264099793Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264099793Cleaning up temporary data from old regions at 1676264099880 (+87 ms)Cleaning up detritus from prior splits at 1676264099880Running coprocessor post-open hooks at 1676264099883 (+3 ms)Region opened successfully at 1676264099883 2023-02-13 04:54:59,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8., pid=78, masterSystemTime=1676264099715 2023-02-13 04:54:59,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:59,885 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:59,885 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:54:59,885 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,885 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,885 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1912): b457ad1e952dae854296990d02485fa8/cf1 is initiating minor compaction (all files) 2023-02-13 04:54:59,885 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1912): b457ad1e952dae854296990d02485fa8/cf2 is initiating minor compaction (all files) 2023-02-13 04:54:59,885 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b457ad1e952dae854296990d02485fa8/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,885 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b457ad1e952dae854296990d02485fa8/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,885 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp, totalSize=35.0 K 2023-02-13 04:54:59,885 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp, totalSize=35.0 K 2023-02-13 04:54:59,886 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264097249 2023-02-13 04:54:59,886 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting 0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264097249 2023-02-13 04:54:59,886 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264097448 2023-02-13 04:54:59,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,886 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264097448 2023-02-13 04:54:59,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b457ad1e952dae854296990d02485fa8, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:54:59,887 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264097624 2023-02-13 04:54:59,887 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264097624 2023-02-13 04:54:59,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264099886"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264099886"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264099886"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264099886"}]},"ts":"1676264099886"} 2023-02-13 04:54:59,887 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264097804 2023-02-13 04:54:59,887 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264097804 2023-02-13 04:54:59,887 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264098018 2023-02-13 04:54:59,888 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264098217 2023-02-13 04:54:59,888 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264098018 2023-02-13 04:54:59,888 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264098422 2023-02-13 04:54:59,888 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264098217 2023-02-13 04:54:59,889 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264098422 2023-02-13 04:54:59,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-13 04:54:59,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure b457ad1e952dae854296990d02485fa8, server=jenkins-hbase4.apache.org,36009,1676264072859 in 321 msec 2023-02-13 04:54:59,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-13 04:54:59,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, ASSIGN in 482 msec 2023-02-13 04:54:59,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a in 926 msec 2023-02-13 04:54:59,903 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] throttle.PressureAwareThroughputController(145): b457ad1e952dae854296990d02485fa8#cf2#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:59,904 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] throttle.PressureAwareThroughputController(145): b457ad1e952dae854296990d02485fa8#cf1#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:59,914 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp/cf2/b9d61c61e5a34fd8914c3b7a285b0178 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/b9d61c61e5a34fd8914c3b7a285b0178 2023-02-13 04:54:59,916 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp/cf1/27ecb1c37b85412cad0afa4355d22432 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/27ecb1c37b85412cad0afa4355d22432 2023-02-13 04:54:59,918 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b457ad1e952dae854296990d02485fa8/cf2 of b457ad1e952dae854296990d02485fa8 into b9d61c61e5a34fd8914c3b7a285b0178(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:59,919 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b457ad1e952dae854296990d02485fa8: 2023-02-13 04:54:59,919 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8., storeName=b457ad1e952dae854296990d02485fa8/cf2, priority=9, startTime=1676264099884; duration=0sec 2023-02-13 04:54:59,919 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,921 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b457ad1e952dae854296990d02485fa8/cf1 of b457ad1e952dae854296990d02485fa8 into 27ecb1c37b85412cad0afa4355d22432(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:54:59,921 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b457ad1e952dae854296990d02485fa8: 2023-02-13 04:54:59,921 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8., storeName=b457ad1e952dae854296990d02485fa8/cf1, priority=9, startTime=1676264099883; duration=0sec 2023-02-13 04:54:59,921 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,938 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:54:59,978 DEBUG [Listener at localhost/33551] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 04:54:59,982 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45712, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 04:54:59,982 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 04:54:59,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HStore(1912): b457ad1e952dae854296990d02485fa8/cf1 is initiating major compaction (all files) 2023-02-13 04:54:59,983 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6cb090e0; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:54:59,983 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b457ad1e952dae854296990d02485fa8/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,984 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 04:54:59,984 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/27ecb1c37b85412cad0afa4355d22432] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp, totalSize=6.8 K 2023-02-13 04:54:59,984 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HStore(1912): b457ad1e952dae854296990d02485fa8/cf2 is initiating major compaction (all files) 2023-02-13 04:54:59,984 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3d940158; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:59,984 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b457ad1e952dae854296990d02485fa8/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:54:59,984 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/b9d61c61e5a34fd8914c3b7a285b0178] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp, totalSize=6.8 K 2023-02-13 04:54:59,984 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 27ecb1c37b85412cad0afa4355d22432, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264097249 2023-02-13 04:54:59,984 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting b9d61c61e5a34fd8914c3b7a285b0178, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264097249 2023-02-13 04:54:59,986 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:54:59,986 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 04:54:59,986 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.HStore(1912): bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 is initiating major compaction (all files) 2023-02-13 04:54:59,986 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@79ee4644; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:54:59,986 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 04:54:59,987 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.HStore(1912): bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 is initiating major compaction (all files) 2023-02-13 04:54:59,987 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@10221006; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 04:54:59,987 INFO [Listener at localhost/33551] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 04:54:59,989 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] throttle.PressureAwareThroughputController(145): b457ad1e952dae854296990d02485fa8#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:59,989 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] throttle.PressureAwareThroughputController(145): b457ad1e952dae854296990d02485fa8#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:54:59,998 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. is not mergeable because it has references 2023-02-13 04:54:59,999 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. is not mergeable because it has references 2023-02-13 04:55:00,001 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp/cf1/9d6c773fabae4a27bb2b7c3d73ea2cc9 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/9d6c773fabae4a27bb2b7c3d73ea2cc9 2023-02-13 04:55:00,001 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/.tmp/cf2/f21ae93c300a407caf34fda2217f5de2 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/f21ae93c300a407caf34fda2217f5de2 2023-02-13 04:55:00,005 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b457ad1e952dae854296990d02485fa8/cf1 of b457ad1e952dae854296990d02485fa8 into 9d6c773fabae4a27bb2b7c3d73ea2cc9(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:55:00,006 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b457ad1e952dae854296990d02485fa8: 2023-02-13 04:55:00,007 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b457ad1e952dae854296990d02485fa8/cf2 of b457ad1e952dae854296990d02485fa8 into f21ae93c300a407caf34fda2217f5de2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:55:00,007 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b457ad1e952dae854296990d02485fa8: 2023-02-13 04:55:00,007 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676264099984; duration=0sec 2023-02-13 04:55:00,007 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676264099983; duration=0sec 2023-02-13 04:55:00,007 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 04:55:00,007 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:55:00,007 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:55:00,008 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:55:00,008 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/f911ceadc70a4eaab45aa42c87203c3e] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp, totalSize=5.0 K 2023-02-13 04:55:00,008 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/4b7819d725d94fb9ac0e101dc4b282d3] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp, totalSize=5.0 K 2023-02-13 04:55:00,008 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] compactions.Compactor(207): Compacting f911ceadc70a4eaab45aa42c87203c3e, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 04:55:00,008 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] compactions.Compactor(207): Compacting 4b7819d725d94fb9ac0e101dc4b282d3, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 04:55:00,011 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] throttle.PressureAwareThroughputController(145): bf5f3b76dbba55abd9f0e0ed2d9ac96a#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:55:00,016 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf5f3b76dbba55abd9f0e0ed2d9ac96a#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:55:00,032 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 900dccfa1be540ab9ff7c5adc1836b82: NONE, but ROW specified in column family configuration 2023-02-13 04:55:00,032 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp/cf2/900dccfa1be540ab9ff7c5adc1836b82 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/900dccfa1be540ab9ff7c5adc1836b82 2023-02-13 04:55:00,033 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e858dc6fd9534a3595f4a326e571e394: NONE, but ROW specified in column family configuration 2023-02-13 04:55:00,034 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/.tmp/cf1/e858dc6fd9534a3595f4a326e571e394 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/e858dc6fd9534a3595f4a326e571e394 2023-02-13 04:55:00,036 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 900dccfa1be540ab9ff7c5adc1836b82: NONE, but ROW specified in column family configuration 2023-02-13 04:55:00,037 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2 of bf5f3b76dbba55abd9f0e0ed2d9ac96a into 900dccfa1be540ab9ff7c5adc1836b82(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:55:00,037 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf5f3b76dbba55abd9f0e0ed2d9ac96a: 2023-02-13 04:55:00,037 INFO [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676264099986; duration=0sec 2023-02-13 04:55:00,037 DEBUG [RS:2;jenkins-hbase4:36009-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:55:00,038 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e858dc6fd9534a3595f4a326e571e394: NONE, but ROW specified in column family configuration 2023-02-13 04:55:00,039 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1 of bf5f3b76dbba55abd9f0e0ed2d9ac96a into e858dc6fd9534a3595f4a326e571e394(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:55:00,039 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf5f3b76dbba55abd9f0e0ed2d9ac96a: 2023-02-13 04:55:00,039 INFO [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676264099986; duration=0sec 2023-02-13 04:55:00,039 DEBUG [RS:2;jenkins-hbase4:36009-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:55:00,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. is not mergeable because it has references 2023-02-13 04:55:00,107 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36009] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. is not mergeable because it has references 2023-02-13 04:55:00,114 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 04:55:00,115 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:55:00,116 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,117 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 04:55:00,121 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264100120"}]},"ts":"1676264100120"} 2023-02-13 04:55:00,122 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 04:55:00,123 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 04:55:00,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, UNASSIGN}] 2023-02-13 04:55:00,124 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, UNASSIGN 2023-02-13 04:55:00,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=bf5f3b76dbba55abd9f0e0ed2d9ac96a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:55:00,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264100125"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264100125"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264100125"}]},"ts":"1676264100125"} 2023-02-13 04:55:00,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure bf5f3b76dbba55abd9f0e0ed2d9ac96a, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:55:00,128 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, UNASSIGN 2023-02-13 04:55:00,128 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b457ad1e952dae854296990d02485fa8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:55:00,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264100128"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264100128"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264100128"}]},"ts":"1676264100128"} 2023-02-13 04:55:00,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure b457ad1e952dae854296990d02485fa8, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:55:00,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 04:55:00,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:55:00,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing bf5f3b76dbba55abd9f0e0ed2d9ac96a, disabling compactions & flushes 2023-02-13 04:55:00,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:55:00,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:55:00,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. after waiting 0 ms 2023-02-13 04:55:00,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:55:00,278 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/4b7819d725d94fb9ac0e101dc4b282d3, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a] to archive 2023-02-13 04:55:00,279 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:55:00,281 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,282 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,283 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,284 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,285 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:55:00,286 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-54727de44d994daeb2938f064e694275 2023-02-13 04:55:00,286 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/4b7819d725d94fb9ac0e101dc4b282d3 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/4b7819d725d94fb9ac0e101dc4b282d3 2023-02-13 04:55:00,287 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-2b50fd89a25446579ea95239b91e387a 2023-02-13 04:55:00,289 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/f911ceadc70a4eaab45aa42c87203c3e, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc] to archive 2023-02-13 04:55:00,290 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:55:00,291 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,292 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,293 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,294 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,294 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:55:00,295 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:55:00,296 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/f911ceadc70a4eaab45aa42c87203c3e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/f911ceadc70a4eaab45aa42c87203c3e 2023-02-13 04:55:00,297 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:55:00,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 04:55:00,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:55:00,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a. 2023-02-13 04:55:00,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for bf5f3b76dbba55abd9f0e0ed2d9ac96a: Waiting for close lock at 1676264100278Running coprocessor pre-close hooks at 1676264100278Disabling compacts and flushes for region at 1676264100278Disabling writes for close at 1676264100278Writing region close event to WAL at 1676264100298 (+20 ms)Running coprocessor post-close hooks at 1676264100301 (+3 ms)Closed at 1676264100302 (+1 ms) 2023-02-13 04:55:00,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:55:00,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b457ad1e952dae854296990d02485fa8 2023-02-13 04:55:00,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b457ad1e952dae854296990d02485fa8, disabling compactions & flushes 2023-02-13 04:55:00,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:55:00,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:55:00,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. after waiting 0 ms 2023-02-13 04:55:00,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:55:00,303 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=bf5f3b76dbba55abd9f0e0ed2d9ac96a, regionState=CLOSED 2023-02-13 04:55:00,303 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264100303"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264100303"}]},"ts":"1676264100303"} 2023-02-13 04:55:00,304 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/27ecb1c37b85412cad0afa4355d22432, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7-bottom] to archive 2023-02-13 04:55:00,305 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:55:00,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 04:55:00,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure bf5f3b76dbba55abd9f0e0ed2d9ac96a, server=jenkins-hbase4.apache.org,36009,1676264072859 in 179 msec 2023-02-13 04:55:00,307 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bf5f3b76dbba55abd9f0e0ed2d9ac96a, UNASSIGN in 183 msec 2023-02-13 04:55:00,308 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/317c03b3799c41bc974cc02d5513a41a.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,309 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/585a3b149d7349ea8b28225213c9ac8e.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,310 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-458f4e2749f2498cb756c4bfe88b8a05 2023-02-13 04:55:00,311 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-a4a0c3435b5747cda1ab348f04a0028a 2023-02-13 04:55:00,312 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-7b68164361124734bb4f1e3b95c4588d 2023-02-13 04:55:00,313 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/27ecb1c37b85412cad0afa4355d22432 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/27ecb1c37b85412cad0afa4355d22432 2023-02-13 04:55:00,314 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/d17d8448721a406793070feb1407d7d7.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,316 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/b9d61c61e5a34fd8914c3b7a285b0178, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab-bottom] to archive 2023-02-13 04:55:00,316 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:55:00,317 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/0a74f0d9cebc43189011aa2d19dda0de.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,318 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c553694bbd224129a9d9dc0d99f88da9.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,319 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,320 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-d38c1b8fdb42486e948adc379d4a4a88 2023-02-13 04:55:00,321 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-664fe50ae4534c86a1dd8fc4e503cfab 2023-02-13 04:55:00,322 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/testSplitTableRegionDeletedRowsDaughter=e4e717d13d69df81025587937cd5f72b-6bf82868aa154b5e8269e13b7c3be6a0 2023-02-13 04:55:00,322 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/b9d61c61e5a34fd8914c3b7a285b0178 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/b9d61c61e5a34fd8914c3b7a285b0178 2023-02-13 04:55:00,323 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab.e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 04:55:00,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:55:00,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8. 2023-02-13 04:55:00,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b457ad1e952dae854296990d02485fa8: Waiting for close lock at 1676264100303Running coprocessor pre-close hooks at 1676264100303Disabling compacts and flushes for region at 1676264100303Disabling writes for close at 1676264100303Writing region close event to WAL at 1676264100324 (+21 ms)Running coprocessor post-close hooks at 1676264100327 (+3 ms)Closed at 1676264100328 (+1 ms) 2023-02-13 04:55:00,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b457ad1e952dae854296990d02485fa8 2023-02-13 04:55:00,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b457ad1e952dae854296990d02485fa8, regionState=CLOSED 2023-02-13 04:55:00,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676264100330"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264100330"}]},"ts":"1676264100330"} 2023-02-13 04:55:00,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 04:55:00,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure b457ad1e952dae854296990d02485fa8, server=jenkins-hbase4.apache.org,36009,1676264072859 in 202 msec 2023-02-13 04:55:00,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 04:55:00,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b457ad1e952dae854296990d02485fa8, UNASSIGN in 208 msec 2023-02-13 04:55:00,334 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264100334"}]},"ts":"1676264100334"} 2023-02-13 04:55:00,335 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 04:55:00,336 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 04:55:00,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 220 msec 2023-02-13 04:55:00,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 04:55:00,421 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 04:55:00,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,424 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 04:55:00,429 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,429 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8 2023-02-13 04:55:00,429 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:55:00,431 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/recovered.edits] 2023-02-13 04:55:00,431 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/recovered.edits] 2023-02-13 04:55:00,431 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/recovered.edits] 2023-02-13 04:55:00,434 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/9d6c773fabae4a27bb2b7c3d73ea2cc9 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf1/9d6c773fabae4a27bb2b7c3d73ea2cc9 2023-02-13 04:55:00,434 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/e858dc6fd9534a3595f4a326e571e394 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf1/e858dc6fd9534a3595f4a326e571e394 2023-02-13 04:55:00,434 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/1d0d112c5bb84d70821d9e30ec8fd0e7 2023-02-13 04:55:00,436 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/2b50fd89a25446579ea95239b91e387a 2023-02-13 04:55:00,437 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/900dccfa1be540ab9ff7c5adc1836b82 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/cf2/900dccfa1be540ab9ff7c5adc1836b82 2023-02-13 04:55:00,437 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/f21ae93c300a407caf34fda2217f5de2 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/cf2/f21ae93c300a407caf34fda2217f5de2 2023-02-13 04:55:00,437 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/317c03b3799c41bc974cc02d5513a41a 2023-02-13 04:55:00,439 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/418946b19cdb436cbcc8474c5d90478a 2023-02-13 04:55:00,440 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/recovered.edits/113.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a/recovered.edits/113.seqid 2023-02-13 04:55:00,440 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/458f4e2749f2498cb756c4bfe88b8a05 2023-02-13 04:55:00,440 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/recovered.edits/113.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8/recovered.edits/113.seqid 2023-02-13 04:55:00,441 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:55:00,441 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b457ad1e952dae854296990d02485fa8 2023-02-13 04:55:00,441 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/54727de44d994daeb2938f064e694275 2023-02-13 04:55:00,443 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/585a3b149d7349ea8b28225213c9ac8e 2023-02-13 04:55:00,444 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/7b68164361124734bb4f1e3b95c4588d 2023-02-13 04:55:00,445 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/a4a0c3435b5747cda1ab348f04a0028a 2023-02-13 04:55:00,446 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf1/d17d8448721a406793070feb1407d7d7 2023-02-13 04:55:00,448 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/0a74f0d9cebc43189011aa2d19dda0de 2023-02-13 04:55:00,450 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/664fe50ae4534c86a1dd8fc4e503cfab 2023-02-13 04:55:00,451 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/668e49c4370a4f07881dbd5e0fe1cb05 2023-02-13 04:55:00,452 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/6bf82868aa154b5e8269e13b7c3be6a0 2023-02-13 04:55:00,453 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/a99ab468a2e74e0ba07280dbbe9ffbf5 2023-02-13 04:55:00,454 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/bcd01a3af22e4c8982b3a894c2a53e1d 2023-02-13 04:55:00,455 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c553694bbd224129a9d9dc0d99f88da9 2023-02-13 04:55:00,456 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/c8d45f624eda4aea8dd4ff7c6cce06ab 2023-02-13 04:55:00,457 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/d38c1b8fdb42486e948adc379d4a4a88 2023-02-13 04:55:00,458 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/cf2/f59eb27353c44b59b49591170a4c5bfc 2023-02-13 04:55:00,460 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/recovered.edits/106.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b/recovered.edits/106.seqid 2023-02-13 04:55:00,460 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e4e717d13d69df81025587937cd5f72b 2023-02-13 04:55:00,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 04:55:00,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,464 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 04:55:00,466 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 04:55:00,466 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,467 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 04:55:00,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264100467"}]},"ts":"9223372036854775807"} 2023-02-13 04:55:00,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264100467"}]},"ts":"9223372036854775807"} 2023-02-13 04:55:00,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264100467"}]},"ts":"9223372036854775807"} 2023-02-13 04:55:00,468 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:55:00,468 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e4e717d13d69df81025587937cd5f72b, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676264096745.e4e717d13d69df81025587937cd5f72b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b457ad1e952dae854296990d02485fa8, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676264098967.b457ad1e952dae854296990d02485fa8.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => bf5f3b76dbba55abd9f0e0ed2d9ac96a, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676264098967.bf5f3b76dbba55abd9f0e0ed2d9ac96a.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 04:55:00,468 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 04:55:00,468 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264100468"}]},"ts":"9223372036854775807"} 2023-02-13 04:55:00,470 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 04:55:00,472 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:55:00,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 50 msec 2023-02-13 04:55:00,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 04:55:00,526 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 04:55:00,542 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=411 (was 406) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:36009-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39236 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39038 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:52482 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:53638 [Waiting for operation #66] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39114 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39088 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39060 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:57976 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:58024 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39078 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:52532 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:58040 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:57982 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39008 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:52488 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-706549021_15 at /127.0.0.1:39066 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 857), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=218 (was 237), ProcessCount=168 (was 168), AvailableMemoryMB=3673 (was 3733) 2023-02-13 04:55:00,556 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=410, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=218, ProcessCount=168, AvailableMemoryMB=3673 2023-02-13 04:55:00,556 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:55:00,558 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:55:00,559 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:55:00,560 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:55:00,561 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:55:00,562 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:55:00,563 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,564 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 empty. 2023-02-13 04:55:00,564 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,564 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 04:55:00,575 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 04:55:00,576 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0ef4bb18d3be3c5b741907872064ff91, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:55:00,584 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:55:00,584 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 0ef4bb18d3be3c5b741907872064ff91, disabling compactions & flushes 2023-02-13 04:55:00,584 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,584 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,584 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. after waiting 0 ms 2023-02-13 04:55:00,584 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,584 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,584 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 0ef4bb18d3be3c5b741907872064ff91: Waiting for close lock at 1676264100584Disabling compacts and flushes for region at 1676264100584Disabling writes for close at 1676264100584Writing region close event to WAL at 1676264100584Closed at 1676264100584 2023-02-13 04:55:00,586 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:55:00,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264100586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264100586"}]},"ts":"1676264100586"} 2023-02-13 04:55:00,588 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:55:00,588 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:55:00,588 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264100588"}]},"ts":"1676264100588"} 2023-02-13 04:55:00,589 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 04:55:00,592 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:55:00,592 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:55:00,592 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:55:00,593 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:55:00,593 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:55:00,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, ASSIGN}] 2023-02-13 04:55:00,594 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, ASSIGN 2023-02-13 04:55:00,595 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=false 2023-02-13 04:55:00,745 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:55:00,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:55:00,748 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264100747"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264100747"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264100747"}]},"ts":"1676264100747"} 2023-02-13 04:55:00,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:55:00,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0ef4bb18d3be3c5b741907872064ff91, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:55:00,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:55:00,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:55:00,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,905 INFO [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,907 DEBUG [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1 2023-02-13 04:55:00,907 DEBUG [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1 2023-02-13 04:55:00,907 INFO [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ef4bb18d3be3c5b741907872064ff91 columnFamilyName cf1 2023-02-13 04:55:00,908 INFO [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] regionserver.HStore(310): Store=0ef4bb18d3be3c5b741907872064ff91/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:55:00,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:00,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:55:00,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0ef4bb18d3be3c5b741907872064ff91; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74501376, jitterRate=0.11015701293945312}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 04:55:00,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0ef4bb18d3be3c5b741907872064ff91: Running coprocessor pre-open hook at 1676264100904Writing region info on filesystem at 1676264100904Initializing all the Stores at 1676264100905 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264100905Cleaning up temporary data from old regions at 1676264100909 (+4 ms)Cleaning up detritus from prior splits at 1676264100910 (+1 ms)Running coprocessor post-open hooks at 1676264100914 (+4 ms)Region opened successfully at 1676264100914 2023-02-13 04:55:00,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91., pid=87, masterSystemTime=1676264100901 2023-02-13 04:55:00,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:00,916 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:55:00,916 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264100916"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264100916"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264100916"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264100916"}]},"ts":"1676264100916"} 2023-02-13 04:55:00,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 04:55:00,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787 in 168 msec 2023-02-13 04:55:00,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 04:55:00,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, ASSIGN in 326 msec 2023-02-13 04:55:00,921 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:55:00,921 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264100921"}]},"ts":"1676264100921"} 2023-02-13 04:55:00,922 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 04:55:00,928 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:55:00,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 369 msec 2023-02-13 04:55:01,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 04:55:01,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:55:01,087 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 04:55:01,088 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 04:55:01,088 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 04:55:01,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 04:55:01,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, REOPEN/MOVE}] 2023-02-13 04:55:01,103 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, REOPEN/MOVE 2023-02-13 04:55:01,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:55:01,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264101103"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264101103"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264101103"}]},"ts":"1676264101103"} 2023-02-13 04:55:01,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:55:01,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0ef4bb18d3be3c5b741907872064ff91, disabling compactions & flushes 2023-02-13 04:55:01,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. after waiting 0 ms 2023-02-13 04:55:01,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 04:55:01,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:55:01,261 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0ef4bb18d3be3c5b741907872064ff91: Waiting for close lock at 1676264101257Running coprocessor pre-close hooks at 1676264101257Disabling compacts and flushes for region at 1676264101257Disabling writes for close at 1676264101258 (+1 ms)Writing region close event to WAL at 1676264101258Running coprocessor post-close hooks at 1676264101261 (+3 ms)Closed at 1676264101261 2023-02-13 04:55:01,261 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 0ef4bb18d3be3c5b741907872064ff91 to self. 2023-02-13 04:55:01,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=CLOSED 2023-02-13 04:55:01,263 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264101263"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264101263"}]},"ts":"1676264101263"} 2023-02-13 04:55:01,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 04:55:01,266 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787 in 160 msec 2023-02-13 04:55:01,267 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46601,1676264072787; forceNewPlan=false, retain=true 2023-02-13 04:55:01,417 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:55:01,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:55:01,417 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264101417"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264101417"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264101417"}]},"ts":"1676264101417"} 2023-02-13 04:55:01,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:55:01,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0ef4bb18d3be3c5b741907872064ff91, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:55:01,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:55:01,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:55:01,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,575 INFO [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,576 DEBUG [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1 2023-02-13 04:55:01,576 DEBUG [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1 2023-02-13 04:55:01,576 INFO [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ef4bb18d3be3c5b741907872064ff91 columnFamilyName cf1 2023-02-13 04:55:01,577 INFO [StoreOpener-0ef4bb18d3be3c5b741907872064ff91-1] regionserver.HStore(310): Store=0ef4bb18d3be3c5b741907872064ff91/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:55:01,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:55:01,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0ef4bb18d3be3c5b741907872064ff91; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62650653, jitterRate=-0.06643252074718475}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 04:55:01,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0ef4bb18d3be3c5b741907872064ff91: Running coprocessor pre-open hook at 1676264101574Writing region info on filesystem at 1676264101574Initializing all the Stores at 1676264101575 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264101575Cleaning up temporary data from old regions at 1676264101579 (+4 ms)Cleaning up detritus from prior splits at 1676264101579Running coprocessor post-open hooks at 1676264101582 (+3 ms)Region opened successfully at 1676264101582 2023-02-13 04:55:01,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91., pid=92, masterSystemTime=1676264101571 2023-02-13 04:55:01,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:55:01,584 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:55:01,584 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264101584"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264101584"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264101584"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264101584"}]},"ts":"1676264101584"} 2023-02-13 04:55:01,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 04:55:01,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787 in 166 msec 2023-02-13 04:55:01,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 04:55:01,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, REOPEN/MOVE in 485 msec 2023-02-13 04:55:01,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 04:55:01,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-02-13 04:55:01,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, ASSIGN}] 2023-02-13 04:55:01,595 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, ASSIGN 2023-02-13 04:55:01,595 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 04:55:01,745 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:55:01,747 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:55:01,747 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:55:01,747 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:55:01,747 DEBUG [jenkins-hbase4:43801] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 04:55:01,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=b88c1467ad2988cf4338d56117f38a07, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:55:01,748 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264101748"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676264101748"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676264101748"}]},"ts":"1676264101748"} 2023-02-13 04:55:01,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure b88c1467ad2988cf4338d56117f38a07, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:55:01,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:55:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b88c1467ad2988cf4338d56117f38a07, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 04:55:01,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:55:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica b88c1467ad2988cf4338d56117f38a07 2023-02-13 04:55:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:55:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b88c1467ad2988cf4338d56117f38a07 2023-02-13 04:55:01,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b88c1467ad2988cf4338d56117f38a07 2023-02-13 04:55:01,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:02,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:03,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,132 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 04:55:04,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:04,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,220 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:55:05,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:05,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:06,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:07,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:08,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:09,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:10,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:11,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:12,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:13,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:14,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:15,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:16,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:17,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:18,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:19,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:20,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:21,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:22,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:23,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:24,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:25,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:26,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:27,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:28,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:29,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:30,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:31,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:32,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:33,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,156 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 04:55:34,163 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 04:55:34,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,194 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 04:55:34,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:34,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:35,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:36,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:37,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:38,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:39,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:40,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:41,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:42,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:43,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:44,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:45,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:46,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:47,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:48,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:49,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:50,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:51,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:52,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:53,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:54,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:55,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:56,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:57,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:58,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:55:59,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:00,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 04:56:01,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 04:56:01,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 04:56:01,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 04:56:01,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:01,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 04:56:02,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:02,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,283 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:03,285 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:03,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:03,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:03,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:03,287 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 0ef4bb18d3be3c5b741907872064ff91, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:03,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 exec-time=5 msec 2023-02-13 04:56:03,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 04:56:03,630 INFO [StoreOpener-b88c1467ad2988cf4338d56117f38a07-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b88c1467ad2988cf4338d56117f38a07 2023-02-13 04:56:03,634 DEBUG [StoreOpener-b88c1467ad2988cf4338d56117f38a07-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1 2023-02-13 04:56:03,634 DEBUG [StoreOpener-b88c1467ad2988cf4338d56117f38a07-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1 2023-02-13 04:56:03,634 INFO [StoreOpener-b88c1467ad2988cf4338d56117f38a07-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b88c1467ad2988cf4338d56117f38a07 columnFamilyName cf1 2023-02-13 04:56:03,635 INFO [StoreOpener-b88c1467ad2988cf4338d56117f38a07-1] regionserver.HStore(310): Store=b88c1467ad2988cf4338d56117f38a07/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:03,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b88c1467ad2988cf4338d56117f38a07; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59758780, jitterRate=-0.10952478647232056}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 04:56:03,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b88c1467ad2988cf4338d56117f38a07: Running coprocessor pre-open hook at 1676264101911Initializing all the Stores at 1676264163630 (+61719 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264163630Running coprocessor post-open hooks at 1676264163637 (+7 ms)Region opened successfully at 1676264163637 2023-02-13 04:56:03,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07., pid=94, masterSystemTime=1676264101904 2023-02-13 04:56:03,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:56:03,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:56:03,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=b88c1467ad2988cf4338d56117f38a07, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:03,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264163638"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676264163638"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676264163638"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676264163638"}]},"ts":"1676264163638"} 2023-02-13 04:56:03,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 04:56:03,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure b88c1467ad2988cf4338d56117f38a07, server=jenkins-hbase4.apache.org,33859,1676264072829 in 1 mins, 1.89 sec 2023-02-13 04:56:03,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 04:56:03,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, ASSIGN in 1 mins, 2.048 sec 2023-02-13 04:56:03,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.573 sec 2023-02-13 04:56:03,646 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676264101066","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62579,"client":"172.31.14.131:53656","queuetimems":0,"class":"HMaster"} 2023-02-13 04:56:03,646 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-13 04:56:05,535 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up 2023-02-13 04:56:05,537 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:05,539 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 04:56:05,542 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264165542"}]},"ts":"1676264165542"} 2023-02-13 04:56:05,544 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 04:56:05,546 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 04:56:05,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, UNASSIGN}] 2023-02-13 04:56:05,547 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, UNASSIGN 2023-02-13 04:56:05,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=b88c1467ad2988cf4338d56117f38a07, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:05,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264165547"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676264165547"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676264165547"}]},"ts":"1676264165547"} 2023-02-13 04:56:05,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure b88c1467ad2988cf4338d56117f38a07, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:05,551 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, UNASSIGN 2023-02-13 04:56:05,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:05,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264165551"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264165551"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264165551"}]},"ts":"1676264165551"} 2023-02-13 04:56:05,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787}] 2023-02-13 04:56:05,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 04:56:05,701 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b88c1467ad2988cf4338d56117f38a07 2023-02-13 04:56:05,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b88c1467ad2988cf4338d56117f38a07, disabling compactions & flushes 2023-02-13 04:56:05,702 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:56:05,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:56:05,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. after waiting 0 ms 2023-02-13 04:56:05,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:56:05,702 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:05,703 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07. 2023-02-13 04:56:05,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b88c1467ad2988cf4338d56117f38a07: Waiting for close lock at 1676264165701Running coprocessor pre-close hooks at 1676264165701Disabling compacts and flushes for region at 1676264165701Disabling writes for close at 1676264165702 (+1 ms)Writing region close event to WAL at 1676264165702Running coprocessor post-close hooks at 1676264165702Closed at 1676264165703 (+1 ms) 2023-02-13 04:56:05,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0ef4bb18d3be3c5b741907872064ff91, disabling compactions & flushes 2023-02-13 04:56:05,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:56:05,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:56:05,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. after waiting 0 ms 2023-02-13 04:56:05,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:56:05,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b88c1467ad2988cf4338d56117f38a07 2023-02-13 04:56:05,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=b88c1467ad2988cf4338d56117f38a07, regionState=CLOSED 2023-02-13 04:56:05,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264165706"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676264165706"}]},"ts":"1676264165706"} 2023-02-13 04:56:05,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 04:56:05,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:05,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91. 2023-02-13 04:56:05,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0ef4bb18d3be3c5b741907872064ff91: Waiting for close lock at 1676264165704Running coprocessor pre-close hooks at 1676264165704Disabling compacts and flushes for region at 1676264165704Disabling writes for close at 1676264165705 (+1 ms)Writing region close event to WAL at 1676264165705Running coprocessor post-close hooks at 1676264165710 (+5 ms)Closed at 1676264165710 2023-02-13 04:56:05,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 04:56:05,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure b88c1467ad2988cf4338d56117f38a07, server=jenkins-hbase4.apache.org,33859,1676264072829 in 159 msec 2023-02-13 04:56:05,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:56:05,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07, UNASSIGN in 165 msec 2023-02-13 04:56:05,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0ef4bb18d3be3c5b741907872064ff91, regionState=CLOSED 2023-02-13 04:56:05,713 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676264165712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264165712"}]},"ts":"1676264165712"} 2023-02-13 04:56:05,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 04:56:05,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 0ef4bb18d3be3c5b741907872064ff91, server=jenkins-hbase4.apache.org,46601,1676264072787 in 162 msec 2023-02-13 04:56:05,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 04:56:05,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0ef4bb18d3be3c5b741907872064ff91, UNASSIGN in 169 msec 2023-02-13 04:56:05,719 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264165719"}]},"ts":"1676264165719"} 2023-02-13 04:56:05,720 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 04:56:05,722 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 04:56:05,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 185 msec 2023-02-13 04:56:05,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 04:56:05,844 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 04:56:05,845 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,847 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,848 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 04:56:05,853 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:56:05,854 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/recovered.edits] 2023-02-13 04:56:05,858 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/recovered.edits/7.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91/recovered.edits/7.seqid 2023-02-13 04:56:05,858 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0ef4bb18d3be3c5b741907872064ff91 2023-02-13 04:56:05,858 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 04:56:05,861 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,863 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 04:56:05,864 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 04:56:05,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 04:56:05,865 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264165865"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:05,865 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264165865"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:05,867 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 04:56:05,867 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0ef4bb18d3be3c5b741907872064ff91, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676264100559.0ef4bb18d3be3c5b741907872064ff91.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b88c1467ad2988cf4338d56117f38a07, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676264100559_0001.b88c1467ad2988cf4338d56117f38a07.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 04:56:05,867 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 04:56:05,867 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264165867"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:05,868 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 04:56:05,871 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:05,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 26 msec 2023-02-13 04:56:05,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 04:56:05,949 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 04:56:05,962 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 410), OpenFileDescriptor=658 (was 857), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=138 (was 218), ProcessCount=168 (was 168), AvailableMemoryMB=3514 (was 3673) 2023-02-13 04:56:05,974 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=138, ProcessCount=168, AvailableMemoryMB=3513 2023-02-13 04:56:05,975 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:05,977 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:56:05,977 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:56:05,978 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:05,979 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:56:05,980 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:56:05,981 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab 2023-02-13 04:56:05,982 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab empty. 2023-02-13 04:56:05,982 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab 2023-02-13 04:56:05,982 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 04:56:05,991 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 04:56:05,992 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 080932bbd490366ebac53303067e83ab, NAME => 'testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:56:05,998 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:05,998 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 080932bbd490366ebac53303067e83ab, disabling compactions & flushes 2023-02-13 04:56:05,998 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:05,998 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:05,998 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. after waiting 0 ms 2023-02-13 04:56:05,998 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:05,998 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:05,998 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 080932bbd490366ebac53303067e83ab: Waiting for close lock at 1676264165998Disabling compacts and flushes for region at 1676264165998Disabling writes for close at 1676264165998Writing region close event to WAL at 1676264165998Closed at 1676264165998 2023-02-13 04:56:06,000 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:56:06,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676264166001"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264166001"}]},"ts":"1676264166001"} 2023-02-13 04:56:06,002 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:56:06,003 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:56:06,003 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264166003"}]},"ts":"1676264166003"} 2023-02-13 04:56:06,004 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 04:56:06,006 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:56:06,007 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:56:06,007 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:56:06,007 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:56:06,007 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:56:06,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, ASSIGN}] 2023-02-13 04:56:06,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, ASSIGN 2023-02-13 04:56:06,009 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36009,1676264072859; forceNewPlan=false, retain=false 2023-02-13 04:56:06,159 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:56:06,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=080932bbd490366ebac53303067e83ab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676264166160"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264166160"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264166160"}]},"ts":"1676264166160"} 2023-02-13 04:56:06,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 080932bbd490366ebac53303067e83ab, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:56:06,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 080932bbd490366ebac53303067e83ab, NAME => 'testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:56:06,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:06,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:06,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,319 INFO [StoreOpener-080932bbd490366ebac53303067e83ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,321 DEBUG [StoreOpener-080932bbd490366ebac53303067e83ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1 2023-02-13 04:56:06,321 DEBUG [StoreOpener-080932bbd490366ebac53303067e83ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1 2023-02-13 04:56:06,321 INFO [StoreOpener-080932bbd490366ebac53303067e83ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 080932bbd490366ebac53303067e83ab columnFamilyName cf1 2023-02-13 04:56:06,322 INFO [StoreOpener-080932bbd490366ebac53303067e83ab-1] regionserver.HStore(310): Store=080932bbd490366ebac53303067e83ab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:06,322 INFO [StoreOpener-080932bbd490366ebac53303067e83ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,323 DEBUG [StoreOpener-080932bbd490366ebac53303067e83ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2 2023-02-13 04:56:06,323 DEBUG [StoreOpener-080932bbd490366ebac53303067e83ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2 2023-02-13 04:56:06,324 INFO [StoreOpener-080932bbd490366ebac53303067e83ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 080932bbd490366ebac53303067e83ab columnFamilyName cf2 2023-02-13 04:56:06,324 INFO [StoreOpener-080932bbd490366ebac53303067e83ab-1] regionserver.HStore(310): Store=080932bbd490366ebac53303067e83ab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:06,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:06,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:06,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:56:06,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 080932bbd490366ebac53303067e83ab; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74626078, jitterRate=0.11201521754264832}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:06,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 080932bbd490366ebac53303067e83ab: Running coprocessor pre-open hook at 1676264166318Writing region info on filesystem at 1676264166318Initializing all the Stores at 1676264166319 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264166319Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264166319Cleaning up temporary data from old regions at 1676264166326 (+7 ms)Cleaning up detritus from prior splits at 1676264166326Running coprocessor post-open hooks at 1676264166330 (+4 ms)Region opened successfully at 1676264166330 2023-02-13 04:56:06,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab., pid=104, masterSystemTime=1676264166314 2023-02-13 04:56:06,332 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,333 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=080932bbd490366ebac53303067e83ab, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,333 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676264166333"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264166333"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264166333"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264166333"}]},"ts":"1676264166333"} 2023-02-13 04:56:06,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 04:56:06,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 080932bbd490366ebac53303067e83ab, server=jenkins-hbase4.apache.org,36009,1676264072859 in 172 msec 2023-02-13 04:56:06,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 04:56:06,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, ASSIGN in 329 msec 2023-02-13 04:56:06,338 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:56:06,338 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264166338"}]},"ts":"1676264166338"} 2023-02-13 04:56:06,339 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 04:56:06,342 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:56:06,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 365 msec 2023-02-13 04:56:06,486 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:06,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 04:56:06,488 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 04:56:06,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 04:56:06,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 04:56:06,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,488 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,490 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,490 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,490 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,490 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,490 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,490 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:06,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,491 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,492 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,492 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,492 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. started... 2023-02-13 04:56:06,493 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:56:06,493 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,494 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,495 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,503 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/5d1a1adc31af448681bbab261633b4df 2023-02-13 04:56:06,514 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/6614891822344d16a1d7f105446ccd18 2023-02-13 04:56:06,517 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/5d1a1adc31af448681bbab261633b4df as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/5d1a1adc31af448681bbab261633b4df 2023-02-13 04:56:06,520 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/5d1a1adc31af448681bbab261633b4df, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:06,521 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/6614891822344d16a1d7f105446ccd18 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/6614891822344d16a1d7f105446ccd18 2023-02-13 04:56:06,524 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/6614891822344d16a1d7f105446ccd18, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:06,525 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 32ms, sequenceid=6, compaction requested=false 2023-02-13 04:56:06,525 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 04:56:06,526 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:06,526 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,526 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:06,526 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:06,526 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,526 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,526 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,528 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,528 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,529 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,529 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 04:56:06,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23a27cfa[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:06,530 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 04:56:06,530 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,532 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,532 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,533 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:06,534 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,534 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,535 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,535 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:56:06,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 04:56:06,537 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 04:56:06,537 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:06,537 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:06,537 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 04:56:06,537 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,540 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,540 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,541 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,541 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,542 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,543 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,543 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,546 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,546 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 04:56:06,546 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:06,545 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:06,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,546 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,547 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,547 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,646 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:06,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:06,658 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:06,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:06,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 04:56:06,659 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 04:56:06,659 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,659 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 04:56:06,660 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 04:56:06,660 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,660 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,661 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,661 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,661 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,661 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,662 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,662 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:06,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,662 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,663 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:06,663 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,663 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:06,663 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,664 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,663 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:06,664 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:06,664 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,664 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. started... 2023-02-13 04:56:06,664 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:06,665 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,672 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,672 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,672 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,672 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,672 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,677 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/fd05b230cc4845d79f950aedffc38b60 2023-02-13 04:56:06,690 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/07c13c07db0b4f27acc03b150e94c846 2023-02-13 04:56:06,693 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/fd05b230cc4845d79f950aedffc38b60 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fd05b230cc4845d79f950aedffc38b60 2023-02-13 04:56:06,697 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fd05b230cc4845d79f950aedffc38b60, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:06,698 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/07c13c07db0b4f27acc03b150e94c846 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/07c13c07db0b4f27acc03b150e94c846 2023-02-13 04:56:06,702 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/07c13c07db0b4f27acc03b150e94c846, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:06,703 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 39ms, sequenceid=19, compaction requested=false 2023-02-13 04:56:06,703 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:06,703 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,703 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:06,703 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:06,703 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,704 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,704 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,705 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,705 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,705 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,705 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 04:56:06,708 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e5b86a0[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:06,708 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 04:56:06,708 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:06,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:06,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,709 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,710 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,710 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:06,710 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,710 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:06,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,711 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,711 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,711 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,711 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,712 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,712 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,712 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,712 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:56:06,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 04:56:06,716 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 04:56:06,716 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:06,716 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:06,716 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 04:56:06,716 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,718 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,719 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,719 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,720 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,722 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,722 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,722 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 04:56:06,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:06,725 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,725 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,726 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:06,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,726 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:06,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,726 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,726 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,727 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,826 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:06,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:06,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:06,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:06,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 04:56:06,840 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 04:56:06,840 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,840 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 04:56:06,840 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 04:56:06,840 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,840 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,841 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,841 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,842 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,842 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:06,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,843 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:06,844 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:06,844 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:06,844 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:06,844 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,844 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. started... 2023-02-13 04:56:06,844 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,844 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,844 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:06,846 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,846 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,847 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,846 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,847 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,847 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,855 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/7dbb7ffb7def450599c0167ed1780533 2023-02-13 04:56:06,866 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/c0c6173284844d86b461c2bde6730412 2023-02-13 04:56:06,869 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/7dbb7ffb7def450599c0167ed1780533 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/7dbb7ffb7def450599c0167ed1780533 2023-02-13 04:56:06,872 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/7dbb7ffb7def450599c0167ed1780533, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:06,873 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/c0c6173284844d86b461c2bde6730412 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/c0c6173284844d86b461c2bde6730412 2023-02-13 04:56:06,876 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/c0c6173284844d86b461c2bde6730412, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:06,877 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 33ms, sequenceid=32, compaction requested=true 2023-02-13 04:56:06,877 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:06,877 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:06,877 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:06,877 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:06,877 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,877 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:06,877 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:06,878 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,878 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,879 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,879 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 04:56:06,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e133d8b[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:06,881 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 04:56:06,881 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,882 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,882 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:06,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:06,882 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:06,885 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,885 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,885 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,886 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,886 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,886 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:06,886 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,886 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,886 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,886 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:56:06,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 04:56:06,889 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 04:56:06,889 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:06,889 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:06,889 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 04:56:06,889 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:06,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,890 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,891 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,891 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,892 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,893 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:06,893 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:06,894 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,900 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 04:56:06,900 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:06,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:06,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:06,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:06,900 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:06,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:06,900 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:06,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:06,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:06,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,001 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:07,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:07,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:07,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:07,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 04:56:07,014 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 04:56:07,014 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,014 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 04:56:07,014 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 04:56:07,015 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,015 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,016 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,016 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,016 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,016 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,017 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,017 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,018 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,018 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,018 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,018 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,018 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:07,018 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,018 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. started... 2023-02-13 04:56:07,019 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:07,019 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,020 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,021 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,021 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,033 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/8fd876ab481f41459ec5865406048b52 2023-02-13 04:56:07,046 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/62ccb0cccf974d99bca34dc6140e3486 2023-02-13 04:56:07,050 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/8fd876ab481f41459ec5865406048b52 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/8fd876ab481f41459ec5865406048b52 2023-02-13 04:56:07,053 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/8fd876ab481f41459ec5865406048b52, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:07,054 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/62ccb0cccf974d99bca34dc6140e3486 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/62ccb0cccf974d99bca34dc6140e3486 2023-02-13 04:56:07,059 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/62ccb0cccf974d99bca34dc6140e3486, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:07,059 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 41ms, sequenceid=45, compaction requested=true 2023-02-13 04:56:07,059 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:07,059 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,060 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,062 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,062 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,063 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 04:56:07,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76ef076a[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:07,064 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 04:56:07,065 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,066 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,066 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,068 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,068 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:56:07,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 04:56:07,071 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 04:56:07,071 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:07,072 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:07,072 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 04:56:07,072 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,075 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,075 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,076 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,077 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,078 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,078 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,080 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,080 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 04:56:07,081 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,081 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:07,080 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,081 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:07,081 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,081 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,081 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,081 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,181 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:07,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:07,192 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:07,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:07,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 04:56:07,194 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 04:56:07,194 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,194 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 04:56:07,194 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 04:56:07,194 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,194 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,196 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,196 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,196 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,196 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,196 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,196 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,197 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,197 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,197 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,197 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,198 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,198 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,198 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,198 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. started... 2023-02-13 04:56:07,198 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:07,199 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,199 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,199 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,201 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,201 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,201 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,206 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/fefc2d202d5b4c05ba39fa74a8235632 2023-02-13 04:56:07,216 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/3a3794d854f54ed1b36b987b31c931db 2023-02-13 04:56:07,220 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/fefc2d202d5b4c05ba39fa74a8235632 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fefc2d202d5b4c05ba39fa74a8235632 2023-02-13 04:56:07,223 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fefc2d202d5b4c05ba39fa74a8235632, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:07,223 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/3a3794d854f54ed1b36b987b31c931db as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/3a3794d854f54ed1b36b987b31c931db 2023-02-13 04:56:07,227 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/3a3794d854f54ed1b36b987b31c931db, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:07,227 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 29ms, sequenceid=58, compaction requested=true 2023-02-13 04:56:07,228 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:07,228 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,228 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,229 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,229 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,230 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 04:56:07,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@461af0e1[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:07,231 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 04:56:07,231 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,233 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,233 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:07,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,233 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:07,238 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,238 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,238 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,241 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,241 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,241 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,241 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,241 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,241 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,241 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:56:07,244 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 04:56:07,244 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 04:56:07,244 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:07,244 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:07,244 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 04:56:07,244 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,246 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,247 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,247 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,249 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,249 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,249 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,251 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,251 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,251 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,251 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 04:56:07,252 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:07,252 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:07,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,353 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:07,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:07,364 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:07,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:07,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 04:56:07,365 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 04:56:07,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 04:56:07,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 04:56:07,365 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,366 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,367 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,367 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,367 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,367 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,368 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,368 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 04:56:07,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,369 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,369 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 04:56:07,369 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,369 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:07,370 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:07,370 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,370 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. started... 2023-02-13 04:56:07,370 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:07,371 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,372 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,380 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/2df70e4dfdc846649c27f3b3f7d6da61 2023-02-13 04:56:07,390 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/0dce5741eacb4d26ba0886d5252e681f 2023-02-13 04:56:07,393 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/2df70e4dfdc846649c27f3b3f7d6da61 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/2df70e4dfdc846649c27f3b3f7d6da61 2023-02-13 04:56:07,396 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/2df70e4dfdc846649c27f3b3f7d6da61, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:07,397 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/0dce5741eacb4d26ba0886d5252e681f as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/0dce5741eacb4d26ba0886d5252e681f 2023-02-13 04:56:07,400 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/0dce5741eacb4d26ba0886d5252e681f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:07,401 INFO [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 30ms, sequenceid=71, compaction requested=true 2023-02-13 04:56:07,401 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:07,401 DEBUG [rs(jenkins-hbase4.apache.org,36009,1676264072859)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 04:56:07,401 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 04:56:07,402 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,402 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,402 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,402 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 04:56:07,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64561004[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:07,404 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 04:56:07,404 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,406 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,406 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,406 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,407 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,407 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:07,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 04:56:07,407 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:07,408 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,408 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 04:56:07,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,36009,1676264072859': 2023-02-13 04:56:07,412 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 04:56:07,412 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 04:56:07,412 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:07,412 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:07,412 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 04:56:07,412 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,414 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,413 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,414 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,414 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:07,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,416 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:07,416 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 04:56:07,416 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:07,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 04:56:07,423 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:07,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 04:56:07,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:07,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:07,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 04:56:07,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 04:56:07,523 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:07,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:07,530 INFO [Listener at localhost/33551] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,531 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.StoreUtils(129): cannot split hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fd05b230cc4845d79f950aedffc38b60 because midkey is the same as first or last row 2023-02-13 04:56:07,531 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.StoreUtils(129): cannot split hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/07c13c07db0b4f27acc03b150e94c846 because midkey is the same as first or last row 2023-02-13 04:56:07,531 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegion(2744): Flushing 080932bbd490366ebac53303067e83ab 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:56:07,540 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/64ff90e8a5fc43698becfb0159e652be 2023-02-13 04:56:07,550 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/4a24b56fa2ed46e9bef3e0ed7209c1c8 2023-02-13 04:56:07,554 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf1/64ff90e8a5fc43698becfb0159e652be as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/64ff90e8a5fc43698becfb0159e652be 2023-02-13 04:56:07,557 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/64ff90e8a5fc43698becfb0159e652be, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:07,557 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/.tmp/cf2/4a24b56fa2ed46e9bef3e0ed7209c1c8 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/4a24b56fa2ed46e9bef3e0ed7209c1c8 2023-02-13 04:56:07,561 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/4a24b56fa2ed46e9bef3e0ed7209c1c8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:07,562 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 080932bbd490366ebac53303067e83ab in 31ms, sequenceid=82, compaction requested=true 2023-02-13 04:56:07,562 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.HRegion(2445): Flush status journal for 080932bbd490366ebac53303067e83ab: 2023-02-13 04:56:07,563 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.StoreUtils(129): cannot split hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fd05b230cc4845d79f950aedffc38b60 because midkey is the same as first or last row 2023-02-13 04:56:07,563 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36009] regionserver.StoreUtils(129): cannot split hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/07c13c07db0b4f27acc03b150e94c846 because midkey is the same as first or last row 2023-02-13 04:56:07,563 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,563 DEBUG [Listener at localhost/33551] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 04:56:07,563 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:07,565 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 04:56:07,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-13 04:56:07,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 04:56:07,568 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264167568"}]},"ts":"1676264167568"} 2023-02-13 04:56:07,569 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 04:56:07,570 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 04:56:07,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, UNASSIGN}] 2023-02-13 04:56:07,571 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, UNASSIGN 2023-02-13 04:56:07,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=080932bbd490366ebac53303067e83ab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:07,572 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676264167572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264167572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264167572"}]},"ts":"1676264167572"} 2023-02-13 04:56:07,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 080932bbd490366ebac53303067e83ab, server=jenkins-hbase4.apache.org,36009,1676264072859}] 2023-02-13 04:56:07,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 04:56:07,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:07,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 080932bbd490366ebac53303067e83ab, disabling compactions & flushes 2023-02-13 04:56:07,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. after waiting 0 ms 2023-02-13 04:56:07,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:56:07,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:07,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab. 2023-02-13 04:56:07,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 080932bbd490366ebac53303067e83ab: Waiting for close lock at 1676264167725Running coprocessor pre-close hooks at 1676264167725Disabling compacts and flushes for region at 1676264167725Disabling writes for close at 1676264167725Writing region close event to WAL at 1676264167726 (+1 ms)Running coprocessor post-close hooks at 1676264167729 (+3 ms)Closed at 1676264167729 2023-02-13 04:56:07,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 080932bbd490366ebac53303067e83ab 2023-02-13 04:56:07,731 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=080932bbd490366ebac53303067e83ab, regionState=CLOSED 2023-02-13 04:56:07,731 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676264167731"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264167731"}]},"ts":"1676264167731"} 2023-02-13 04:56:07,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 04:56:07,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 080932bbd490366ebac53303067e83ab, server=jenkins-hbase4.apache.org,36009,1676264072859 in 159 msec 2023-02-13 04:56:07,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 04:56:07,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=080932bbd490366ebac53303067e83ab, UNASSIGN in 163 msec 2023-02-13 04:56:07,735 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264167735"}]},"ts":"1676264167735"} 2023-02-13 04:56:07,736 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 04:56:07,738 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 04:56:07,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 173 msec 2023-02-13 04:56:07,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 04:56:07,870 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 04:56:07,871 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-13 04:56:07,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,873 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,873 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 04:56:07,876 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab 2023-02-13 04:56:07,878 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/recovered.edits] 2023-02-13 04:56:07,881 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/2df70e4dfdc846649c27f3b3f7d6da61 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/2df70e4dfdc846649c27f3b3f7d6da61 2023-02-13 04:56:07,882 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/5d1a1adc31af448681bbab261633b4df to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/5d1a1adc31af448681bbab261633b4df 2023-02-13 04:56:07,883 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/64ff90e8a5fc43698becfb0159e652be to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/64ff90e8a5fc43698becfb0159e652be 2023-02-13 04:56:07,883 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/7dbb7ffb7def450599c0167ed1780533 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/7dbb7ffb7def450599c0167ed1780533 2023-02-13 04:56:07,884 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/8fd876ab481f41459ec5865406048b52 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/8fd876ab481f41459ec5865406048b52 2023-02-13 04:56:07,885 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fd05b230cc4845d79f950aedffc38b60 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fd05b230cc4845d79f950aedffc38b60 2023-02-13 04:56:07,886 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fefc2d202d5b4c05ba39fa74a8235632 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf1/fefc2d202d5b4c05ba39fa74a8235632 2023-02-13 04:56:07,888 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/07c13c07db0b4f27acc03b150e94c846 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/07c13c07db0b4f27acc03b150e94c846 2023-02-13 04:56:07,889 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/0dce5741eacb4d26ba0886d5252e681f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/0dce5741eacb4d26ba0886d5252e681f 2023-02-13 04:56:07,891 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/3a3794d854f54ed1b36b987b31c931db to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/3a3794d854f54ed1b36b987b31c931db 2023-02-13 04:56:07,891 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/4a24b56fa2ed46e9bef3e0ed7209c1c8 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/4a24b56fa2ed46e9bef3e0ed7209c1c8 2023-02-13 04:56:07,893 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/62ccb0cccf974d99bca34dc6140e3486 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/62ccb0cccf974d99bca34dc6140e3486 2023-02-13 04:56:07,894 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/6614891822344d16a1d7f105446ccd18 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/6614891822344d16a1d7f105446ccd18 2023-02-13 04:56:07,894 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/c0c6173284844d86b461c2bde6730412 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/cf2/c0c6173284844d86b461c2bde6730412 2023-02-13 04:56:07,897 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/recovered.edits/85.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab/recovered.edits/85.seqid 2023-02-13 04:56:07,898 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testInvalidSplitKey/080932bbd490366ebac53303067e83ab 2023-02-13 04:56:07,898 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 04:56:07,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,901 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 04:56:07,904 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 04:56:07,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 04:56:07,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264167905"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:07,906 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 04:56:07,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 080932bbd490366ebac53303067e83ab, NAME => 'testInvalidSplitKey,,1676264165977.080932bbd490366ebac53303067e83ab.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 04:56:07,906 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 04:56:07,906 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264167906"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:07,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 04:56:07,909 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:07,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 38 msec 2023-02-13 04:56:07,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 04:56:07,974 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 04:56:07,988 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=369 (was 349) - Thread LEAK? -, OpenFileDescriptor=741 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=127 (was 138), ProcessCount=168 (was 168), AvailableMemoryMB=3509 (was 3513) 2023-02-13 04:56:08,000 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=369, OpenFileDescriptor=741, MaxFileDescriptor=60000, SystemLoadAverage=127, ProcessCount=168, AvailableMemoryMB=3509 2023-02-13 04:56:08,001 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:08,003 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:56:08,003 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:56:08,004 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:08,005 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:56:08,006 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:56:08,007 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,008 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 empty. 2023-02-13 04:56:08,008 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,008 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 04:56:08,017 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 04:56:08,018 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2f8c3e17a349695e2d1507baff20f938, NAME => 'testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:56:08,024 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:08,024 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 2f8c3e17a349695e2d1507baff20f938, disabling compactions & flushes 2023-02-13 04:56:08,024 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,024 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,024 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. after waiting 0 ms 2023-02-13 04:56:08,024 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,024 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,024 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 2f8c3e17a349695e2d1507baff20f938: Waiting for close lock at 1676264168024Disabling compacts and flushes for region at 1676264168024Disabling writes for close at 1676264168024Writing region close event to WAL at 1676264168024Closed at 1676264168024 2023-02-13 04:56:08,026 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:56:08,026 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264168026"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264168026"}]},"ts":"1676264168026"} 2023-02-13 04:56:08,027 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:56:08,028 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:56:08,028 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264168028"}]},"ts":"1676264168028"} 2023-02-13 04:56:08,029 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 04:56:08,031 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:56:08,032 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:56:08,032 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:56:08,032 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:56:08,032 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:56:08,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, ASSIGN}] 2023-02-13 04:56:08,034 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, ASSIGN 2023-02-13 04:56:08,034 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:08,184 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:56:08,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=2f8c3e17a349695e2d1507baff20f938, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264168185"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264168185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264168185"}]},"ts":"1676264168185"} 2023-02-13 04:56:08,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:08,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2f8c3e17a349695e2d1507baff20f938, NAME => 'testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:56:08,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:08,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:08,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,342 INFO [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,344 DEBUG [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1 2023-02-13 04:56:08,344 DEBUG [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1 2023-02-13 04:56:08,344 INFO [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f8c3e17a349695e2d1507baff20f938 columnFamilyName cf1 2023-02-13 04:56:08,344 INFO [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] regionserver.HStore(310): Store=2f8c3e17a349695e2d1507baff20f938/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:08,345 INFO [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,346 DEBUG [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2 2023-02-13 04:56:08,346 DEBUG [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2 2023-02-13 04:56:08,346 INFO [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f8c3e17a349695e2d1507baff20f938 columnFamilyName cf2 2023-02-13 04:56:08,346 INFO [StoreOpener-2f8c3e17a349695e2d1507baff20f938-1] regionserver.HStore(310): Store=2f8c3e17a349695e2d1507baff20f938/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:08,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:08,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:08,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:56:08,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2f8c3e17a349695e2d1507baff20f938; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70103005, jitterRate=0.044616177678108215}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:08,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2f8c3e17a349695e2d1507baff20f938: Running coprocessor pre-open hook at 1676264168342Writing region info on filesystem at 1676264168342Initializing all the Stores at 1676264168342Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264168342Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264168342Cleaning up temporary data from old regions at 1676264168348 (+6 ms)Cleaning up detritus from prior splits at 1676264168348Running coprocessor post-open hooks at 1676264168351 (+3 ms)Region opened successfully at 1676264168351 2023-02-13 04:56:08,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938., pid=111, masterSystemTime=1676264168338 2023-02-13 04:56:08,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=2f8c3e17a349695e2d1507baff20f938, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,353 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264168353"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264168353"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264168353"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264168353"}]},"ts":"1676264168353"} 2023-02-13 04:56:08,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 04:56:08,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 in 167 msec 2023-02-13 04:56:08,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 04:56:08,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, ASSIGN in 324 msec 2023-02-13 04:56:08,358 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:56:08,358 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264168358"}]},"ts":"1676264168358"} 2023-02-13 04:56:08,359 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 04:56:08,361 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:56:08,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 357 msec 2023-02-13 04:56:08,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:08,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 04:56:08,513 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 04:56:08,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:56:08,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,513 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,514 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,514 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,514 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,514 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,515 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,515 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:08,516 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:08,516 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. started... 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,518 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,519 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:56:08,519 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,519 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,519 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,520 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,521 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,521 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,521 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,529 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/95a895b5384446dca772a020a30667bb 2023-02-13 04:56:08,544 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/8854a15f9c3a46ed869bdf01a679476e 2023-02-13 04:56:08,548 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/95a895b5384446dca772a020a30667bb as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb 2023-02-13 04:56:08,551 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:08,552 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/8854a15f9c3a46ed869bdf01a679476e as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e 2023-02-13 04:56:08,556 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:08,556 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 37ms, sequenceid=6, compaction requested=false 2023-02-13 04:56:08,557 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,557 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f8c3e17a349695e2d1507baff20f938: 2023-02-13 04:56:08,557 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,557 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,560 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,560 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 04:56:08,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63ae8660[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:08,562 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:56:08,562 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,563 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,563 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,563 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,564 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,564 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:08,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:08,565 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,565 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,565 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,565 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,565 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,565 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,565 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,566 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,566 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,566 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:08,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:56:08,569 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 04:56:08,569 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:08,569 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:08,569 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,569 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:08,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,571 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,572 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,572 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,572 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,573 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,574 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,574 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,575 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,577 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:08,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,577 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:08,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,678 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:08,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:08,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:08,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:08,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 04:56:08,690 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,690 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,690 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 04:56:08,690 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:56:08,691 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,691 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,692 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,692 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,692 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:08,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:08,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,695 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,697 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,697 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,697 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,698 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:08,698 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. started... 2023-02-13 04:56:08,698 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,706 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/87f4d32424e4496ba8216d14f17235f5 2023-02-13 04:56:08,715 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/0494d17e89c24abd99598b259654fc8c 2023-02-13 04:56:08,718 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/87f4d32424e4496ba8216d14f17235f5 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 2023-02-13 04:56:08,721 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:08,722 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/0494d17e89c24abd99598b259654fc8c as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c 2023-02-13 04:56:08,725 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:08,726 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 28ms, sequenceid=19, compaction requested=false 2023-02-13 04:56:08,726 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f8c3e17a349695e2d1507baff20f938: 2023-02-13 04:56:08,726 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,726 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:08,726 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:08,726 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,726 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,726 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,730 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,730 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,730 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,730 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 04:56:08,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6df40928[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:08,732 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:56:08,732 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:08,733 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:08,733 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:08,733 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:08,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,734 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,736 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:08,739 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:56:08,739 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 04:56:08,739 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:08,739 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:08,739 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,739 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:08,740 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,740 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,740 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,740 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,740 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,741 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,741 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,742 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,743 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,744 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,744 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,746 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,746 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,747 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:08,746 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:08,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,747 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,847 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:08,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:08,859 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:08,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:08,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 04:56:08,861 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,861 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,861 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 04:56:08,861 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:56:08,861 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,861 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,863 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,863 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,864 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:08,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:08,865 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,865 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:08,865 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,865 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,865 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:08,865 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,865 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. started... 2023-02-13 04:56:08,865 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:08,867 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,867 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,873 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/e1309aeb210449f8babffad82a48b1ba 2023-02-13 04:56:08,883 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/162d433c39244607ab54a480811abac2 2023-02-13 04:56:08,886 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/e1309aeb210449f8babffad82a48b1ba as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba 2023-02-13 04:56:08,890 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:08,890 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/162d433c39244607ab54a480811abac2 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 2023-02-13 04:56:08,893 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:08,894 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 29ms, sequenceid=32, compaction requested=true 2023-02-13 04:56:08,894 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f8c3e17a349695e2d1507baff20f938: 2023-02-13 04:56:08,894 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:08,894 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,896 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 04:56:08,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cf8718f[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:08,898 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:56:08,898 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,899 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:08,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,900 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:08,900 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:08,900 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:08,900 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:08,900 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,901 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,901 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:08,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:08,904 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:56:08,904 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 04:56:08,904 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:08,904 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:08,904 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 04:56:08,904 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:08,906 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:08,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:08,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,907 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,908 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,908 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:08,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:08,910 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,910 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,910 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,916 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:08,916 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:08,916 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:08,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:08,917 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,916 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:08,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,917 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,917 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:08,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:08,917 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:08,917 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:08,917 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,017 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:09,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:09,029 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:09,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:09,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 04:56:09,031 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:09,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 04:56:09,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:56:09,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,032 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,032 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,032 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,032 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,033 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,033 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,034 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,035 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:09,035 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:09,035 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:09,035 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. started... 2023-02-13 04:56:09,034 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,035 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:09,036 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,038 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,038 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,038 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,038 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,038 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,043 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/165c59e9653946c692b7d48459e37820 2023-02-13 04:56:09,053 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/ab5dc335db004d4390df874a311d8ffa 2023-02-13 04:56:09,056 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/165c59e9653946c692b7d48459e37820 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 2023-02-13 04:56:09,059 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:09,059 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/ab5dc335db004d4390df874a311d8ffa as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa 2023-02-13 04:56:09,063 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:09,063 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 28ms, sequenceid=45, compaction requested=true 2023-02-13 04:56:09,063 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f8c3e17a349695e2d1507baff20f938: 2023-02-13 04:56:09,063 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:09,063 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:09,064 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:09,064 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,064 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,064 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,065 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,065 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,065 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,065 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 04:56:09,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28fa9824[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:09,067 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:56:09,067 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,068 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,068 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,068 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,068 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:09,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,068 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,068 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:09,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,069 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,069 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,069 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:09,070 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,070 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,070 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,072 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,072 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,072 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,072 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:09,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:56:09,075 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 04:56:09,075 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:09,075 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:09,075 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,075 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:09,076 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,076 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,076 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,076 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,076 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,076 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,077 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,077 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,078 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,078 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,080 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,080 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,081 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:09,085 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:09,084 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,084 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:09,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,085 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,185 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:09,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:09,197 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:09,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:09,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 04:56:09,198 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:09,198 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,198 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 04:56:09,198 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:56:09,198 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,199 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,201 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,201 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,201 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,201 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,202 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,202 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:09,203 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:09,203 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. started... 2023-02-13 04:56:09,204 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:09,204 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,205 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,205 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,205 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,205 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,205 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,211 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/01ec973c10ea4569bc9f67afcca4d9b1 2023-02-13 04:56:09,221 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/6159188346394f5fb9b71b0d0fd86f35 2023-02-13 04:56:09,224 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/01ec973c10ea4569bc9f67afcca4d9b1 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 2023-02-13 04:56:09,227 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:09,228 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/6159188346394f5fb9b71b0d0fd86f35 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 2023-02-13 04:56:09,231 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:09,231 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 27ms, sequenceid=58, compaction requested=true 2023-02-13 04:56:09,231 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f8c3e17a349695e2d1507baff20f938: 2023-02-13 04:56:09,231 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:09,231 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:09,232 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:09,232 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,232 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,232 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,237 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,237 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,237 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,237 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 04:56:09,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f7096dd[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:09,239 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:56:09,239 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,240 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:09,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,240 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,241 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,241 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:09,241 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,241 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,241 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,243 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,243 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:09,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:56:09,246 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 04:56:09,246 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:09,246 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:09,246 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,246 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,248 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,249 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,250 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,250 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,250 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:09,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:09,253 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,253 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,354 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:09,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:09,365 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:09,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:09,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 04:56:09,366 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:09,366 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,366 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 04:56:09,366 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 04:56:09,367 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,367 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,369 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,369 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,369 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,369 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,369 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,369 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:09,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:09,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,371 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 04:56:09,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,371 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,371 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:09,371 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:09,371 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. started... 2023-02-13 04:56:09,371 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:09,372 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,372 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,373 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,373 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,373 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,373 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,379 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 2023-02-13 04:56:09,390 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/d655eefaf7194b0595268a92810bb98e 2023-02-13 04:56:09,393 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 2023-02-13 04:56:09,397 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:09,398 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/d655eefaf7194b0595268a92810bb98e as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e 2023-02-13 04:56:09,401 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:09,402 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 31ms, sequenceid=71, compaction requested=true 2023-02-13 04:56:09,402 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f8c3e17a349695e2d1507baff20f938: 2023-02-13 04:56:09,402 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:09,402 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:09,402 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:09,402 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,403 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 04:56:09,403 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 04:56:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,404 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,404 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:09,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 04:56:09,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72807552[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:09,406 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 04:56:09,406 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:09,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:09,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,408 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,408 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 04:56:09,408 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,408 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 04:56:09,408 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:09,408 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:09,410 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,410 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,410 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,410 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,411 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,411 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,411 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 04:56:09,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 04:56:09,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:09,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 04:56:09,414 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 04:56:09,414 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:09,414 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:09,414 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 04:56:09,414 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:09,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,417 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,417 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,418 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,419 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,419 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:09,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 04:56:09,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 04:56:09,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:09,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:09,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:09,426 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:09,426 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:09,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:09,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:09,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:09,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 04:56:09,524 ERROR [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid a51c43df-067e-41c1-a591-2b7ca0d868cc) service to localhost/127.0.0.1:44231 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:56:09,526 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:09,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:09,785 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 04:56:09,786 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:09,787 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:09,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:09,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:09,788 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:09,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:09,789 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:09,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:09,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:10,037 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,037 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:10,037 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:10,037 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:10,038 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 04:56:10,038 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:10,038 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:10,038 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:10,038 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:10,039 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:10,040 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 04:56:10,040 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:10,041 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:10,041 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:10,041 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:10,041 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,041 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:10,041 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,042 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,042 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,042 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:10,042 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,042 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,042 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:10,043 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,044 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,044 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,044 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:10,044 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,044 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,044 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,045 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,045 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:10,045 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,045 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,045 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,046 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,046 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,046 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:10,046 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,046 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:10,046 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,046 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,046 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,046 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 04:56:10,047 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:10,047 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:10,048 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264200048 2023-02-13 04:56:10,049 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:10,049 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,051 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:10,052 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:10,052 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:10,052 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264230052 2023-02-13 04:56:10,052 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264290052 2023-02-13 04:56:10,052 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:10,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:10,054 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:10,054 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:10,054 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:10,302 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,302 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:10,302 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:10,302 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:10,303 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:10,303 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:10,303 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:10,303 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:10,303 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:10,304 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:10,305 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 04:56:10,305 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:10,305 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:10,305 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,306 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:10,307 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,307 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,307 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,307 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:10,307 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:10,307 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,308 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,309 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,310 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,310 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,310 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:10,310 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,310 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:10,310 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,310 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,310 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,310 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:56:10,311 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:10,311 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:10,312 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264200312 2023-02-13 04:56:10,313 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:10,313 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,315 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:10,316 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:10,316 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:10,316 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264230316 2023-02-13 04:56:10,316 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264290316 2023-02-13 04:56:10,316 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:10,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:10,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN}] 2023-02-13 04:56:10,318 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:10,318 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:10,318 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:10,566 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,566 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:10,566 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:10,566 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:10,567 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:10,567 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:10,567 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:10,567 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:10,567 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:10,567 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:10,568 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 04:56:10,569 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:10,569 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:10,569 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:10,570 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:10,570 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,570 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:10,570 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,570 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,571 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,571 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:10,571 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,571 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,571 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,572 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:10,572 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:10,572 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,572 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,572 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,572 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:10,573 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,573 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,573 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,573 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:10,573 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,574 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,574 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,574 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,574 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:10,574 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,575 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,575 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,575 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,575 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,576 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:10,576 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,576 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:10,576 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,576 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,576 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,577 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 04:56:10,577 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:10,578 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:10,578 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264200578 2023-02-13 04:56:10,580 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:10,580 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,582 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:10,583 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:10,583 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:10,583 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264230583 2023-02-13 04:56:10,583 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264290583 2023-02-13 04:56:10,583 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:56:10,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN}] 2023-02-13 04:56:10,583 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:10,585 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:10,586 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:10,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264170586"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264170586"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264170586"}]},"ts":"1676264170586"} 2023-02-13 04:56:10,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:10,739 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2f8c3e17a349695e2d1507baff20f938, disabling compactions & flushes 2023-02-13 04:56:10,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:10,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:10,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. after waiting 0 ms 2023-02-13 04:56:10,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:10,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2f8c3e17a349695e2d1507baff20f938 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:56:10,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/976bc2e13abc4e28be769652053e3d96 2023-02-13 04:56:10,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/dcbf6c40254943e59005ae0dfdfc75c9 2023-02-13 04:56:10,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf1/976bc2e13abc4e28be769652053e3d96 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 2023-02-13 04:56:10,765 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:10,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/.tmp/cf2/dcbf6c40254943e59005ae0dfdfc75c9 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 2023-02-13 04:56:10,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:10,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 29ms, sequenceid=82, compaction requested=true 2023-02-13 04:56:10,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:56:10,774 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:10,775 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. 2023-02-13 04:56:10,775 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2f8c3e17a349695e2d1507baff20f938: Waiting for close lock at 1676264170739Running coprocessor pre-close hooks at 1676264170739Disabling compacts and flushes for region at 1676264170739Disabling writes for close at 1676264170740 (+1 ms)Obtaining lock to block concurrent updates at 1676264170740Preparing flush snapshotting stores in 2f8c3e17a349695e2d1507baff20f938 at 1676264170740Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676264170740Flushing stores of testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. at 1676264170740Flushing 2f8c3e17a349695e2d1507baff20f938/cf1: creating writer at 1676264170741 (+1 ms)Flushing 2f8c3e17a349695e2d1507baff20f938/cf1: appending metadata at 1676264170743 (+2 ms)Flushing 2f8c3e17a349695e2d1507baff20f938/cf1: closing flushed file at 1676264170743Flushing 2f8c3e17a349695e2d1507baff20f938/cf2: creating writer at 1676264170752 (+9 ms)Flushing 2f8c3e17a349695e2d1507baff20f938/cf2: appending metadata at 1676264170753 (+1 ms)Flushing 2f8c3e17a349695e2d1507baff20f938/cf2: closing flushed file at 1676264170753Flushing 2f8c3e17a349695e2d1507baff20f938/cf1: reopening flushed file at 1676264170762 (+9 ms)Flushing 2f8c3e17a349695e2d1507baff20f938/cf2: reopening flushed file at 1676264170766 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2f8c3e17a349695e2d1507baff20f938 in 29ms, sequenceid=82, compaction requested=true at 1676264170769 (+3 ms)Writing region close event to WAL at 1676264170770 (+1 ms)Running coprocessor post-close hooks at 1676264170774 (+4 ms)Closed at 1676264170775 (+1 ms) 2023-02-13 04:56:10,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,777 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED 2023-02-13 04:56:10,777 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264170777"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264170777"}]},"ts":"1676264170777"} 2023-02-13 04:56:10,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 04:56:10,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 in 191 msec 2023-02-13 04:56:10,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 04:56:10,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN in 197 msec 2023-02-13 04:56:10,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:10,786 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:10,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:10,833 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,833 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:10,833 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:10,833 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:10,833 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:10,834 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:10,834 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:10,834 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:10,834 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:10,834 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:10,835 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 04:56:10,835 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:10,836 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:10,836 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:10,836 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:10,836 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:10,836 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:10,837 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,838 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:10,838 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,838 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:10,838 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:10,838 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,838 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,839 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:10,840 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,840 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,840 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:10,840 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,841 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:10,841 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,841 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,841 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:10,841 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:10,842 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,842 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:10,842 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:10,842 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:10,842 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,842 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,842 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,842 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:10,842 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:10,842 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 04:56:10,843 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:10,843 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:10,844 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264200844 2023-02-13 04:56:10,844 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:10,845 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:10,847 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:10,847 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:10,847 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:10,847 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264230847 2023-02-13 04:56:10,847 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264290847 2023-02-13 04:56:10,848 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:10,850 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:10,859 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=2f8c3e17a349695e2d1507baff20f938, threads=14 2023-02-13 04:56:10,859 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,859 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,859 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,862 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,863 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,863 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,863 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,863 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,866 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,867 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,867 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,867 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,868 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:10,868 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,028 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,028 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,034 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,050 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,050 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,056 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,064 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,079 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,088 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,090 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,090 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,093 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,093 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,103 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,103 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 2f8c3e17a349695e2d1507baff20f938 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:56:11,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:11,168 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:11,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:11,241 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:56:11,348 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,348 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:11,348 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:11,348 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:11,348 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:11,349 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:11,349 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:11,349 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:11,349 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:11,350 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:11,351 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 04:56:11,351 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:11,351 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:11,351 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:11,351 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:11,352 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:11,352 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:11,352 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:11,352 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:11,352 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:11,352 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:11,353 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:11,353 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:11,353 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:11,353 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:11,353 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:11,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:11,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:11,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:11,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:11,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:11,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:11,355 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:11,356 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:11,356 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:11,356 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:11,356 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:11,356 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:11,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:11,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:11,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:11,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:11,357 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:11,358 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:11,358 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,358 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,358 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,358 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:11,358 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:11,358 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 04:56:11,358 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:11,359 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:11,359 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264201359 2023-02-13 04:56:11,360 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:11,361 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,363 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:11,363 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:11,363 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:11,363 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264231363 2023-02-13 04:56:11,363 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264291363 2023-02-13 04:56:11,363 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:11,371 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=2f8c3e17a349695e2d1507baff20f938, threads=14 2023-02-13 04:56:11,372 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,372 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,373 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,373 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,373 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,373 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,373 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,378 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,378 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,378 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,378 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,378 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,380 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,380 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,471 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,476 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,476 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,476 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,483 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,484 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,484 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,485 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,486 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,489 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,490 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,491 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,492 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,855 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 for region: 2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:11,855 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 2f8c3e17a349695e2d1507baff20f938 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:56:11,884 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:11,888 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:56:11,890 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:56:11,890 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:11,890 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:11,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:11,890 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:12,114 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:12,114 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:12,115 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:12,116 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 04:56:12,116 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:12,116 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:12,116 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,117 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,118 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,119 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,120 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,120 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,120 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:12,120 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,120 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,120 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:12,121 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,121 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,121 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,121 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,122 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,122 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:56:12,122 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:12,123 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:12,123 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264202123 2023-02-13 04:56:12,124 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:12,124 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,126 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:12,127 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:12,127 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:12,127 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264232127 2023-02-13 04:56:12,127 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264292127 2023-02-13 04:56:12,127 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:12,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:12,131 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:12,131 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:12,131 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:12,377 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,377 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:12,377 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:12,377 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:12,377 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:12,378 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:12,378 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:12,378 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:12,378 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:12,378 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:12,379 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 04:56:12,379 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:12,379 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:12,380 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:12,380 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:12,380 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:12,380 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:12,380 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,380 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,381 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:12,382 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,383 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,383 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,383 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,383 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:12,383 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,383 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:12,384 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:12,384 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,384 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,385 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,385 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,385 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,385 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:56:12,385 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:12,386 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:12,386 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264202386 2023-02-13 04:56:12,387 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,387 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:12,390 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:12,390 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:12,390 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:12,390 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264232390 2023-02-13 04:56:12,390 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264292390 2023-02-13 04:56:12,390 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:56:12,390 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:12,392 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264172392"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676264172392"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676264172392"}]},"ts":"1676264172392"} 2023-02-13 04:56:12,392 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264172392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264172392"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264172392"}]},"ts":"1676264172392"} 2023-02-13 04:56:12,393 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264172392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264172392"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264172392"}]},"ts":"1676264172392"} 2023-02-13 04:56:12,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:12,394 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:12,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:12,538 ERROR [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid 82110bba-6b14-4557-aa3b-b536cbd539dd) service to localhost/127.0.0.1:44231 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:12,641 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:12,641 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:12,642 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:12,643 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 04:56:12,643 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:12,643 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:12,643 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:12,643 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,644 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,645 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:12,645 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,645 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,645 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,645 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,646 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,647 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,647 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:12,647 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,647 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,647 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,648 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:12,649 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,649 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:12,649 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:12,649 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:12,649 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,649 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,649 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,649 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,649 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,649 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 04:56:12,650 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:12,650 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:12,651 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264202651 2023-02-13 04:56:12,653 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,653 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:12,655 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:12,655 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:12,655 INFO [Listener at localhost/33551] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:12,655 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 04:56:12,655 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=909ab782e42393951fb0e57d4b83d533, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 04:56:12,655 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:12,656 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264232655 2023-02-13 04:56:12,656 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264292656 2023-02-13 04:56:12,656 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:12,656 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676264172656"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676264172656"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676264172656"}]},"ts":"1676264172656"} 2023-02-13 04:56:12,656 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264172656"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264172656"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264172656"}]},"ts":"1676264172656"} 2023-02-13 04:56:12,656 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264172656"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264172656"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264172656"}]},"ts":"1676264172656"} 2023-02-13 04:56:12,658 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:12,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:12,659 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:12,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:12,906 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:12,906 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:12,907 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:12,907 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:12,908 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 04:56:12,908 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:12,908 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:12,909 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:12,909 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:12,909 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:12,909 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:12,909 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,910 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,911 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,912 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:12,913 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:12,914 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,914 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:12,914 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:12,914 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:12,914 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,914 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,914 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,914 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,914 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:12,914 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:56:12,915 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:12,915 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:12,916 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264202916 2023-02-13 04:56:12,917 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:12,918 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,920 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:12,920 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:12,920 INFO [Listener at localhost/33551] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:12,920 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 04:56:12,921 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=909ab782e42393951fb0e57d4b83d533, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 04:56:12,921 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:12,921 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264232921 2023-02-13 04:56:12,921 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264292921 2023-02-13 04:56:12,921 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 04:56:12,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:12,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN}] 2023-02-13 04:56:12,923 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:12,923 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:12,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:12,923 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:13,171 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,171 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:13,171 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:13,171 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:13,172 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:13,172 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:13,172 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:13,172 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:13,172 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:13,173 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:13,174 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:13,175 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:13,176 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,177 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,178 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:13,179 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,179 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,179 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,179 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,179 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,179 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 04:56:13,179 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:13,180 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:13,180 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264203180 2023-02-13 04:56:13,182 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,182 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:13,184 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:13,184 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:13,184 INFO [Listener at localhost/33551] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:13,184 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 04:56:13,184 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=909ab782e42393951fb0e57d4b83d533, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 04:56:13,184 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:13,184 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264233184 2023-02-13 04:56:13,184 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264293184 2023-02-13 04:56:13,185 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 04:56:13,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN}] 2023-02-13 04:56:13,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:13,187 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN 2023-02-13 04:56:13,187 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:13,188 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN 2023-02-13 04:56:13,188 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:13,337 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:56:13,339 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=909ab782e42393951fb0e57d4b83d533, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264173339"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264173339"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264173339"}]},"ts":"1676264173339"} 2023-02-13 04:56:13,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 909ab782e42393951fb0e57d4b83d533, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:13,342 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,342 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264173342"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264173342"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264173342"}]},"ts":"1676264173342"} 2023-02-13 04:56:13,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 953f4d722d17e0cb1a46ddd56a04bf8e, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:13,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 909ab782e42393951fb0e57d4b83d533, NAME => 'testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 04:56:13,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:13,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:13,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,496 INFO [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,496 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1 2023-02-13 04:56:13,496 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1 2023-02-13 04:56:13,497 INFO [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 909ab782e42393951fb0e57d4b83d533 columnFamilyName cf1 2023-02-13 04:56:13,507 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1-top 2023-02-13 04:56:13,510 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820-top 2023-02-13 04:56:13,513 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5-top 2023-02-13 04:56:13,516 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb-top 2023-02-13 04:56:13,519 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96-top 2023-02-13 04:56:13,521 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba-top 2023-02-13 04:56:13,524 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5-top 2023-02-13 04:56:13,524 INFO [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(310): Store=909ab782e42393951fb0e57d4b83d533/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:13,524 INFO [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,525 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2 2023-02-13 04:56:13,525 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2 2023-02-13 04:56:13,525 INFO [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 909ab782e42393951fb0e57d4b83d533 columnFamilyName cf2 2023-02-13 04:56:13,536 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c-top 2023-02-13 04:56:13,539 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2-top 2023-02-13 04:56:13,542 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35-top 2023-02-13 04:56:13,545 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e-top 2023-02-13 04:56:13,548 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa-top 2023-02-13 04:56:13,552 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e-top 2023-02-13 04:56:13,554 DEBUG [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9-top 2023-02-13 04:56:13,555 INFO [StoreOpener-909ab782e42393951fb0e57d4b83d533-1] regionserver.HStore(310): Store=909ab782e42393951fb0e57d4b83d533/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:13,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:13,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 909ab782e42393951fb0e57d4b83d533; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64004097, jitterRate=-0.046264633536338806}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:13,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 909ab782e42393951fb0e57d4b83d533: Running coprocessor pre-open hook at 1676264173495Writing region info on filesystem at 1676264173495Initializing all the Stores at 1676264173495Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264173495Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264173495Cleaning up temporary data from old regions at 1676264173557 (+62 ms)Cleaning up detritus from prior splits at 1676264173558 (+1 ms)Running coprocessor post-open hooks at 1676264173561 (+3 ms)Region opened successfully at 1676264173561 2023-02-13 04:56:13,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533., pid=117, masterSystemTime=1676264173492 2023-02-13 04:56:13,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:13,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:56:13,562 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:13,563 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:13,563 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,563 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): 909ab782e42393951fb0e57d4b83d533/cf1 is initiating minor compaction (all files) 2023-02-13 04:56:13,563 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,563 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): 909ab782e42393951fb0e57d4b83d533/cf2 is initiating minor compaction (all files) 2023-02-13 04:56:13,563 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 909ab782e42393951fb0e57d4b83d533/cf1 in testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,563 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 909ab782e42393951fb0e57d4b83d533/cf2 in testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,563 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96-top] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/.tmp, totalSize=34.9 K 2023-02-13 04:56:13,563 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9-top] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/.tmp, totalSize=34.9 K 2023-02-13 04:56:13,564 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264168507 2023-02-13 04:56:13,564 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264168507 2023-02-13 04:56:13,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:13,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,565 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264168679 2023-02-13 04:56:13,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 953f4d722d17e0cb1a46ddd56a04bf8e, NAME => 'testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 04:56:13,564 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264168679 2023-02-13 04:56:13,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=909ab782e42393951fb0e57d4b83d533, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,565 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264168849 2023-02-13 04:56:13,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:13,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264173565"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264173565"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264173565"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264173565"}]},"ts":"1676264173565"} 2023-02-13 04:56:13,565 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264168849 2023-02-13 04:56:13,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:13,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,565 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264169018 2023-02-13 04:56:13,566 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264169018 2023-02-13 04:56:13,566 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676264169187 2023-02-13 04:56:13,566 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676264169187 2023-02-13 04:56:13,566 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676264169355 2023-02-13 04:56:13,567 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676264169355 2023-02-13 04:56:13,567 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264169528 2023-02-13 04:56:13,567 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264169528 2023-02-13 04:56:13,569 INFO [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-13 04:56:13,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 909ab782e42393951fb0e57d4b83d533, server=jenkins-hbase4.apache.org,33859,1676264072829 in 226 msec 2023-02-13 04:56:13,570 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1 2023-02-13 04:56:13,570 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1 2023-02-13 04:56:13,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN in 384 msec 2023-02-13 04:56:13,570 INFO [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 953f4d722d17e0cb1a46ddd56a04bf8e columnFamilyName cf1 2023-02-13 04:56:13,580 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): 909ab782e42393951fb0e57d4b83d533#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:13,580 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): 909ab782e42393951fb0e57d4b83d533#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:13,591 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1-bottom 2023-02-13 04:56:13,594 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820-bottom 2023-02-13 04:56:13,596 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/.tmp/cf1/30d9e918d6ec410a88e8df134e2978b1 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/30d9e918d6ec410a88e8df134e2978b1 2023-02-13 04:56:13,597 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5-bottom 2023-02-13 04:56:13,598 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/.tmp/cf2/e71db79716804ca8aa1ef083579e4666 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/e71db79716804ca8aa1ef083579e4666 2023-02-13 04:56:13,600 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 909ab782e42393951fb0e57d4b83d533/cf1 of 909ab782e42393951fb0e57d4b83d533 into 30d9e918d6ec410a88e8df134e2978b1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:13,600 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 909ab782e42393951fb0e57d4b83d533: 2023-02-13 04:56:13,600 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533., storeName=909ab782e42393951fb0e57d4b83d533/cf1, priority=9, startTime=1676264173561; duration=0sec 2023-02-13 04:56:13,601 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:13,602 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb-bottom 2023-02-13 04:56:13,603 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 909ab782e42393951fb0e57d4b83d533/cf2 of 909ab782e42393951fb0e57d4b83d533 into e71db79716804ca8aa1ef083579e4666(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:13,603 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 909ab782e42393951fb0e57d4b83d533: 2023-02-13 04:56:13,603 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533., storeName=909ab782e42393951fb0e57d4b83d533/cf2, priority=9, startTime=1676264173561; duration=0sec 2023-02-13 04:56:13,603 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:13,605 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96-bottom 2023-02-13 04:56:13,608 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba-bottom 2023-02-13 04:56:13,610 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5-bottom 2023-02-13 04:56:13,610 INFO [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(310): Store=953f4d722d17e0cb1a46ddd56a04bf8e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:13,610 INFO [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,611 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2 2023-02-13 04:56:13,611 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2 2023-02-13 04:56:13,611 INFO [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 953f4d722d17e0cb1a46ddd56a04bf8e columnFamilyName cf2 2023-02-13 04:56:13,622 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c-bottom 2023-02-13 04:56:13,624 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2-bottom 2023-02-13 04:56:13,627 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35-bottom 2023-02-13 04:56:13,629 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e-bottom 2023-02-13 04:56:13,631 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa-bottom 2023-02-13 04:56:13,634 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e-bottom 2023-02-13 04:56:13,636 DEBUG [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9-bottom 2023-02-13 04:56:13,637 INFO [StoreOpener-953f4d722d17e0cb1a46ddd56a04bf8e-1] regionserver.HStore(310): Store=953f4d722d17e0cb1a46ddd56a04bf8e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:13,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:13,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:13,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 953f4d722d17e0cb1a46ddd56a04bf8e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65463679, jitterRate=-0.024515166878700256}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:13,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 953f4d722d17e0cb1a46ddd56a04bf8e: Running coprocessor pre-open hook at 1676264173565Writing region info on filesystem at 1676264173566 (+1 ms)Initializing all the Stores at 1676264173568 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264173568Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264173568Cleaning up temporary data from old regions at 1676264173638 (+70 ms)Cleaning up detritus from prior splits at 1676264173638Running coprocessor post-open hooks at 1676264173640 (+2 ms)Region opened successfully at 1676264173640 2023-02-13 04:56:13,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e., pid=118, masterSystemTime=1676264173492 2023-02-13 04:56:13,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:13,642 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:13,642 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:13,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:56:13,642 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,642 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,642 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): 953f4d722d17e0cb1a46ddd56a04bf8e/cf1 is initiating minor compaction (all files) 2023-02-13 04:56:13,642 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): 953f4d722d17e0cb1a46ddd56a04bf8e/cf2 is initiating minor compaction (all files) 2023-02-13 04:56:13,642 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 953f4d722d17e0cb1a46ddd56a04bf8e/cf1 in testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,642 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 953f4d722d17e0cb1a46ddd56a04bf8e/cf2 in testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,643 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/.tmp, totalSize=34.9 K 2023-02-13 04:56:13,643 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/.tmp, totalSize=34.9 K 2023-02-13 04:56:13,643 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264168507 2023-02-13 04:56:13,643 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264168507 2023-02-13 04:56:13,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:13,644 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264168679 2023-02-13 04:56:13,644 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264168679 2023-02-13 04:56:13,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264173644"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264173644"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264173644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264173644"}]},"ts":"1676264173644"} 2023-02-13 04:56:13,644 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264168849 2023-02-13 04:56:13,644 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264168849 2023-02-13 04:56:13,644 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264169018 2023-02-13 04:56:13,644 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264169018 2023-02-13 04:56:13,645 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264169187 2023-02-13 04:56:13,645 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264169187 2023-02-13 04:56:13,645 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264169355 2023-02-13 04:56:13,645 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264169355 2023-02-13 04:56:13,646 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264169528 2023-02-13 04:56:13,646 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264169528 2023-02-13 04:56:13,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-13 04:56:13,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 953f4d722d17e0cb1a46ddd56a04bf8e, server=jenkins-hbase4.apache.org,33859,1676264072829 in 302 msec 2023-02-13 04:56:13,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-13 04:56:13,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN in 462 msec 2023-02-13 04:56:13,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 04:56:13,649 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:13,649 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 04:56:13,656 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): 953f4d722d17e0cb1a46ddd56a04bf8e#cf2#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:13,656 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): 953f4d722d17e0cb1a46ddd56a04bf8e#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:13,666 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/.tmp/cf2/f26fbe057e9948eeb2dca189b9d2bb24 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/f26fbe057e9948eeb2dca189b9d2bb24 2023-02-13 04:56:13,666 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/.tmp/cf1/35fcfd25e2994cb0b521885442d03dce as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/35fcfd25e2994cb0b521885442d03dce 2023-02-13 04:56:13,669 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 953f4d722d17e0cb1a46ddd56a04bf8e/cf2 of 953f4d722d17e0cb1a46ddd56a04bf8e into f26fbe057e9948eeb2dca189b9d2bb24(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:13,669 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 953f4d722d17e0cb1a46ddd56a04bf8e/cf1 of 953f4d722d17e0cb1a46ddd56a04bf8e into 35fcfd25e2994cb0b521885442d03dce(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:13,669 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 953f4d722d17e0cb1a46ddd56a04bf8e: 2023-02-13 04:56:13,669 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 953f4d722d17e0cb1a46ddd56a04bf8e: 2023-02-13 04:56:13,670 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e., storeName=953f4d722d17e0cb1a46ddd56a04bf8e/cf2, priority=9, startTime=1676264173642; duration=0sec 2023-02-13 04:56:13,670 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e., storeName=953f4d722d17e0cb1a46ddd56a04bf8e/cf1, priority=9, startTime=1676264173641; duration=0sec 2023-02-13 04:56:13,670 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:13,670 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:13,685 ERROR [Listener at localhost/33551] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 04:56:13,685 INFO [Listener at localhost/33551] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 04:56:13,686 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-13 04:56:13,687 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=3480cf0bddc2adc7fa210c7f8e2c70b4, daughterA=86ee5aad653c7d36bbe7f423e8fa84f5, daughterB=e5e5be97dcb4e9a93e2c911fecc4d10b 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:13,688 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 04:56:13,689 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:13,689 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=8ad8f350a2b6f56d4128dc02f389ea94, daughterA=cc7e469ee5eb3889f5d7044238390067, daughterB=a6deada879885dca6c4b89e4a7394b55 2023-02-13 04:56:13,689 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:13,689 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 04:56:13,689 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:13,689 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3bcdfe2dbd4e99a75b6c62f3d21619a6, daughterA=84e3a6e0598265f86960fec92a925b18, daughterB=a7f3a8cf991d76c5fb7b296292b15b64 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3bcdfe2dbd4e99a75b6c62f3d21619a6, ASSIGN 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=ec38915ce1c4d08b104c37dce39eedc0, daughterA=2dece162b7a5eb9a13add441b9de6ecb, daughterB=3071776f1f3f38a1657d8b0a287d44a1 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e4e717d13d69df81025587937cd5f72b, daughterA=b457ad1e952dae854296990d02485fa8, daughterB=bf5f3b76dbba55abd9f0e0ed2d9ac96a 2023-02-13 04:56:13,690 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRollbackForSplitTableRegionWithReplica, region=b88c1467ad2988cf4338d56117f38a07 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0ef4bb18d3be3c5b741907872064ff91, daughterA=92b988f2b9ec128533b35118dc5659aa, daughterB=029319db4de33e4a883adfd212240757 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:13,691 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:13,692 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 04:56:13,692 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 909ab782e42393951fb0e57d4b83d533, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 953f4d722d17e0cb1a46ddd56a04bf8e, server=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 held the lock before restarting, call acquireLock to restore it. 2023-02-13 04:56:13,693 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,693 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,693 INFO [Listener at localhost/33551] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2f8c3e17a349695e2d1507baff20f938, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,693 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 909ab782e42393951fb0e57d4b83d533, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,694 DEBUG [Listener at localhost/33551] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 953f4d722d17e0cb1a46ddd56a04bf8e, server=jenkins-hbase4.apache.org,33859,1676264072829 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 04:56:13,694 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 04:56:13,694 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 04:56:13,695 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829, table=hbase:meta, region=1588230740 2023-02-13 04:56:13,695 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676264203695 2023-02-13 04:56:13,696 DEBUG [Listener at localhost/33551] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 04:56:13,696 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:13,699 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=dc413a9db9894ddca28879b9c07ec853, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36009,1676264072859, regionLocation=jenkins-hbase4.apache.org,36009,1676264072859, openSeqNum=2 2023-02-13 04:56:13,699 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=2f8c3e17a349695e2d1507baff20f938, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=2 2023-02-13 04:56:13,699 INFO [Listener at localhost/33551] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33859,1676264072829, table=testRecoveryAndDoubleExecution, region=2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:13,699 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=86 2023-02-13 04:56:13,700 DEBUG [Listener at localhost/33551] assignment.RegionStateStore(147): Load hbase:meta entry region=909ab782e42393951fb0e57d4b83d533, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33859,1676264072829, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829, openSeqNum=86 2023-02-13 04:56:13,700 INFO [Listener at localhost/33551] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 04:56:13,700 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676264233700 2023-02-13 04:56:13,700 INFO [Listener at localhost/33551] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676264293700 2023-02-13 04:56:13,700 INFO [Listener at localhost/33551] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 04:56:13,700 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 04:56:13,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2f8c3e17a349695e2d1507baff20f938, daughterA=953f4d722d17e0cb1a46ddd56a04bf8e, daughterB=909ab782e42393951fb0e57d4b83d533 in 3.9140 sec 2023-02-13 04:56:13,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33859] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:50508 deadline: 1676264233951, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938. is not online on jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:14,062 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:56:14,067 INFO [Listener at localhost/33551] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 04:56:14,068 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:56:14,077 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:14,079 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 04:56:14,079 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-13 04:56:14,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 04:56:14,082 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264174082"}]},"ts":"1676264174082"} 2023-02-13 04:56:14,083 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 04:56:14,084 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 04:56:14,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, UNASSIGN}] 2023-02-13 04:56:14,085 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, UNASSIGN 2023-02-13 04:56:14,085 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=909ab782e42393951fb0e57d4b83d533, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:14,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264174085"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264174085"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264174085"}]},"ts":"1676264174085"} 2023-02-13 04:56:14,086 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 909ab782e42393951fb0e57d4b83d533, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:14,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, UNASSIGN 2023-02-13 04:56:14,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:14,089 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264174088"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264174088"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264174088"}]},"ts":"1676264174088"} 2023-02-13 04:56:14,089 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 953f4d722d17e0cb1a46ddd56a04bf8e, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:14,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 04:56:14,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:14,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 953f4d722d17e0cb1a46ddd56a04bf8e, disabling compactions & flushes 2023-02-13 04:56:14,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:14,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:14,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. after waiting 0 ms 2023-02-13 04:56:14,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:14,239 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96-bottom] to archive 2023-02-13 04:56:14,239 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:14,241 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,242 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,243 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,243 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,244 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,245 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,246 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,247 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9-bottom] to archive 2023-02-13 04:56:14,248 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:14,249 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,250 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,251 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,252 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,253 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,254 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,255 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:56:14,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:14,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e. 2023-02-13 04:56:14,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 953f4d722d17e0cb1a46ddd56a04bf8e: Waiting for close lock at 1676264174238Running coprocessor pre-close hooks at 1676264174238Disabling compacts and flushes for region at 1676264174238Disabling writes for close at 1676264174238Writing region close event to WAL at 1676264174255 (+17 ms)Running coprocessor post-close hooks at 1676264174258 (+3 ms)Closed at 1676264174258 2023-02-13 04:56:14,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:14,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:14,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 909ab782e42393951fb0e57d4b83d533, disabling compactions & flushes 2023-02-13 04:56:14,261 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:14,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:14,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. after waiting 0 ms 2023-02-13 04:56:14,261 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:14,261 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=953f4d722d17e0cb1a46ddd56a04bf8e, regionState=CLOSED 2023-02-13 04:56:14,261 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264174261"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264174261"}]},"ts":"1676264174261"} 2023-02-13 04:56:14,262 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96-top] to archive 2023-02-13 04:56:14,262 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:14,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 04:56:14,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 953f4d722d17e0cb1a46ddd56a04bf8e, server=jenkins-hbase4.apache.org,33859,1676264072829 in 173 msec 2023-02-13 04:56:14,264 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=953f4d722d17e0cb1a46ddd56a04bf8e, UNASSIGN in 180 msec 2023-02-13 04:56:14,265 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,266 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,267 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,268 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,268 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,269 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,271 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9-top] to archive 2023-02-13 04:56:14,271 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:14,273 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,273 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,274 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,275 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,276 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,277 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,278 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:56:14,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:14,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533. 2023-02-13 04:56:14,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 909ab782e42393951fb0e57d4b83d533: Waiting for close lock at 1676264174260Running coprocessor pre-close hooks at 1676264174260Disabling compacts and flushes for region at 1676264174260Disabling writes for close at 1676264174261 (+1 ms)Writing region close event to WAL at 1676264174279 (+18 ms)Running coprocessor post-close hooks at 1676264174281 (+2 ms)Closed at 1676264174281 2023-02-13 04:56:14,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:14,283 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=909ab782e42393951fb0e57d4b83d533, regionState=CLOSED 2023-02-13 04:56:14,283 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676264174283"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264174283"}]},"ts":"1676264174283"} 2023-02-13 04:56:14,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 04:56:14,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 909ab782e42393951fb0e57d4b83d533, server=jenkins-hbase4.apache.org,33859,1676264072829 in 199 msec 2023-02-13 04:56:14,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-13 04:56:14,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=909ab782e42393951fb0e57d4b83d533, UNASSIGN in 202 msec 2023-02-13 04:56:14,288 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264174288"}]},"ts":"1676264174288"} 2023-02-13 04:56:14,288 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 04:56:14,290 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 04:56:14,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 211 msec 2023-02-13 04:56:14,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 04:56:14,384 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 04:56:14,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-13 04:56:14,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,387 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,387 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 04:56:14,392 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,392 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:14,393 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:14,402 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/recovered.edits] 2023-02-13 04:56:14,403 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/recovered.edits] 2023-02-13 04:56:14,407 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/.regioninfo to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/.regioninfo 2023-02-13 04:56:14,407 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/01ec973c10ea4569bc9f67afcca4d9b1 2023-02-13 04:56:14,408 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/recovered.edits] 2023-02-13 04:56:14,409 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/165c59e9653946c692b7d48459e37820 2023-02-13 04:56:14,410 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,410 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/87f4d32424e4496ba8216d14f17235f5 2023-02-13 04:56:14,412 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,412 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/95a895b5384446dca772a020a30667bb 2023-02-13 04:56:14,412 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/.regioninfo to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/.regioninfo 2023-02-13 04:56:14,413 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,413 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/976bc2e13abc4e28be769652053e3d96 2023-02-13 04:56:14,414 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,415 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/e1309aeb210449f8babffad82a48b1ba 2023-02-13 04:56:14,416 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,416 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/01ec973c10ea4569bc9f67afcca4d9b1.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,416 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5 2023-02-13 04:56:14,417 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,417 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/165c59e9653946c692b7d48459e37820.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,418 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,419 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/87f4d32424e4496ba8216d14f17235f5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,422 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/0494d17e89c24abd99598b259654fc8c 2023-02-13 04:56:14,423 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/95a895b5384446dca772a020a30667bb.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,424 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/162d433c39244607ab54a480811abac2 2023-02-13 04:56:14,425 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/976bc2e13abc4e28be769652053e3d96.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,425 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,425 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/6159188346394f5fb9b71b0d0fd86f35 2023-02-13 04:56:14,426 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/e1309aeb210449f8babffad82a48b1ba.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,426 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,427 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/8854a15f9c3a46ed869bdf01a679476e 2023-02-13 04:56:14,427 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf1/ffc01b9f4e0a4f6388db7b7f265d9ba5.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,427 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,429 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/ab5dc335db004d4390df874a311d8ffa 2023-02-13 04:56:14,429 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,430 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/d655eefaf7194b0595268a92810bb98e 2023-02-13 04:56:14,431 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/0494d17e89c24abd99598b259654fc8c.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,432 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/cf2/dcbf6c40254943e59005ae0dfdfc75c9 2023-02-13 04:56:14,433 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/162d433c39244607ab54a480811abac2.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,433 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,434 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/6159188346394f5fb9b71b0d0fd86f35.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/recovered.edits/85.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938/recovered.edits/85.seqid 2023-02-13 04:56:14,436 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/8854a15f9c3a46ed869bdf01a679476e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,436 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/35fcfd25e2994cb0b521885442d03dce to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf1/35fcfd25e2994cb0b521885442d03dce 2023-02-13 04:56:14,438 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/ab5dc335db004d4390df874a311d8ffa.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,440 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/d655eefaf7194b0595268a92810bb98e.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,440 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/f26fbe057e9948eeb2dca189b9d2bb24 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/cf2/f26fbe057e9948eeb2dca189b9d2bb24 2023-02-13 04:56:14,441 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/909ab782e42393951fb0e57d4b83d533/cf2/dcbf6c40254943e59005ae0dfdfc75c9.2f8c3e17a349695e2d1507baff20f938 2023-02-13 04:56:14,443 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e/recovered.edits/90.seqid 2023-02-13 04:56:14,443 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/30d9e918d6ec410a88e8df134e2978b1 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf1/30d9e918d6ec410a88e8df134e2978b1 2023-02-13 04:56:14,444 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/953f4d722d17e0cb1a46ddd56a04bf8e 2023-02-13 04:56:14,446 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/e71db79716804ca8aa1ef083579e4666 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/cf2/e71db79716804ca8aa1ef083579e4666 2023-02-13 04:56:14,448 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533/recovered.edits/90.seqid 2023-02-13 04:56:14,449 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testRecoveryAndDoubleExecution/909ab782e42393951fb0e57d4b83d533 2023-02-13 04:56:14,449 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 04:56:14,451 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,453 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 04:56:14,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 04:56:14,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 04:56:14,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264174455"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:14,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264174455"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:14,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264174455"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:14,457 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:56:14,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2f8c3e17a349695e2d1507baff20f938, NAME => 'testRecoveryAndDoubleExecution,,1676264168004.2f8c3e17a349695e2d1507baff20f938.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 953f4d722d17e0cb1a46ddd56a04bf8e, NAME => 'testRecoveryAndDoubleExecution,,1676264169786.953f4d722d17e0cb1a46ddd56a04bf8e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 909ab782e42393951fb0e57d4b83d533, NAME => 'testRecoveryAndDoubleExecution,41,1676264169786.909ab782e42393951fb0e57d4b83d533.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 04:56:14,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 04:56:14,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264174457"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:14,458 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 04:56:14,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 04:56:14,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 76 msec 2023-02-13 04:56:14,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 04:56:14,489 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 04:56:14,504 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 369) - Thread LEAK? -, OpenFileDescriptor=843 (was 741) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=117 (was 127), ProcessCount=168 (was 168), AvailableMemoryMB=3482 (was 3509) 2023-02-13 04:56:14,517 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=843, MaxFileDescriptor=60000, SystemLoadAverage=117, ProcessCount=168, AvailableMemoryMB=3482 2023-02-13 04:56:14,517 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:14,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:56:14,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:56:14,522 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:14,523 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:56:14,524 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:56:14,525 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,525 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d empty. 2023-02-13 04:56:14,526 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,526 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 04:56:14,534 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 04:56:14,535 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 244360b45c972a590dcc9cdd6803198d, NAME => 'testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:56:14,542 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:14,542 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 244360b45c972a590dcc9cdd6803198d, disabling compactions & flushes 2023-02-13 04:56:14,542 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,542 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,542 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. after waiting 0 ms 2023-02-13 04:56:14,542 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,542 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,542 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 244360b45c972a590dcc9cdd6803198d: Waiting for close lock at 1676264174542Disabling compacts and flushes for region at 1676264174542Disabling writes for close at 1676264174542Writing region close event to WAL at 1676264174542Closed at 1676264174542 2023-02-13 04:56:14,544 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:56:14,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264174544"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264174544"}]},"ts":"1676264174544"} 2023-02-13 04:56:14,546 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:56:14,546 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:56:14,546 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264174546"}]},"ts":"1676264174546"} 2023-02-13 04:56:14,547 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 04:56:14,549 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:56:14,550 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:56:14,550 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:56:14,550 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:56:14,550 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:56:14,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, ASSIGN}] 2023-02-13 04:56:14,551 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, ASSIGN 2023-02-13 04:56:14,552 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:14,702 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:56:14,703 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=244360b45c972a590dcc9cdd6803198d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:14,703 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264174703"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264174703"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264174703"}]},"ts":"1676264174703"} 2023-02-13 04:56:14,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 244360b45c972a590dcc9cdd6803198d, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:14,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 244360b45c972a590dcc9cdd6803198d, NAME => 'testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:56:14,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:14,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:14,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,858 INFO [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,860 DEBUG [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1 2023-02-13 04:56:14,860 DEBUG [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1 2023-02-13 04:56:14,860 INFO [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 244360b45c972a590dcc9cdd6803198d columnFamilyName cf1 2023-02-13 04:56:14,860 INFO [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] regionserver.HStore(310): Store=244360b45c972a590dcc9cdd6803198d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:14,860 INFO [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,861 DEBUG [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2 2023-02-13 04:56:14,861 DEBUG [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2 2023-02-13 04:56:14,862 INFO [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 244360b45c972a590dcc9cdd6803198d columnFamilyName cf2 2023-02-13 04:56:14,862 INFO [StoreOpener-244360b45c972a590dcc9cdd6803198d-1] regionserver.HStore(310): Store=244360b45c972a590dcc9cdd6803198d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:14,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:14,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:14,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:56:14,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 244360b45c972a590dcc9cdd6803198d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69291659, jitterRate=0.032526180148124695}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:14,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 244360b45c972a590dcc9cdd6803198d: Running coprocessor pre-open hook at 1676264174858Writing region info on filesystem at 1676264174858Initializing all the Stores at 1676264174858Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264174858Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264174858Cleaning up temporary data from old regions at 1676264174863 (+5 ms)Cleaning up detritus from prior splits at 1676264174864 (+1 ms)Running coprocessor post-open hooks at 1676264174867 (+3 ms)Region opened successfully at 1676264174867 2023-02-13 04:56:14,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d., pid=127, masterSystemTime=1676264174855 2023-02-13 04:56:14,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:14,869 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=244360b45c972a590dcc9cdd6803198d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:14,869 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264174869"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264174869"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264174869"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264174869"}]},"ts":"1676264174869"} 2023-02-13 04:56:14,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 04:56:14,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 244360b45c972a590dcc9cdd6803198d, server=jenkins-hbase4.apache.org,33859,1676264072829 in 166 msec 2023-02-13 04:56:14,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 04:56:14,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, ASSIGN in 322 msec 2023-02-13 04:56:14,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:56:14,873 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264174873"}]},"ts":"1676264174873"} 2023-02-13 04:56:14,874 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 04:56:14,876 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:56:14,877 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 355 msec 2023-02-13 04:56:15,029 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:15,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,030 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 04:56:15,030 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 04:56:15,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,031 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,033 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,033 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,033 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,033 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,034 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,034 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,035 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,035 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,036 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:15,036 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:15,036 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,036 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. started... 2023-02-13 04:56:15,037 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:56:15,037 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,038 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,038 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,038 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,038 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,038 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,045 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/78c5c6a80efe4b3380352496b657a656 2023-02-13 04:56:15,055 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/4f6cdedaebbf486a84bd578702fb6800 2023-02-13 04:56:15,058 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/78c5c6a80efe4b3380352496b657a656 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656 2023-02-13 04:56:15,061 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:15,061 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/4f6cdedaebbf486a84bd578702fb6800 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800 2023-02-13 04:56:15,064 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:15,065 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 28ms, sequenceid=6, compaction requested=false 2023-02-13 04:56:15,065 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,065 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 244360b45c972a590dcc9cdd6803198d: 2023-02-13 04:56:15,066 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:15,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:15,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,066 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,067 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,067 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,067 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,067 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 04:56:15,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d4b1684[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:15,069 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 04:56:15,069 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,071 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,071 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,071 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,072 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:15,072 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:15,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,074 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,074 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,074 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,074 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:15,076 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:56:15,076 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 04:56:15,076 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:15,076 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:15,076 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,077 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,078 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,079 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,079 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,081 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,081 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,081 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,087 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,087 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,087 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:15,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:15,087 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,188 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:15,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:15,199 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:15,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:15,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,200 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,200 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,200 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 04:56:15,200 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 04:56:15,201 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,201 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,201 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,202 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,202 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,201 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,202 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,202 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,202 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,203 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,204 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,204 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,204 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:15,204 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:15,204 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,204 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. started... 2023-02-13 04:56:15,204 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:15,205 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,205 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,205 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,205 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,205 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,205 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,212 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/88bbfde6097b47b1a218117edc2de36f 2023-02-13 04:56:15,221 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/548d76689eb94fb687138e6c7a80d849 2023-02-13 04:56:15,224 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/88bbfde6097b47b1a218117edc2de36f as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f 2023-02-13 04:56:15,227 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:15,228 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/548d76689eb94fb687138e6c7a80d849 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849 2023-02-13 04:56:15,230 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:15,231 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 27ms, sequenceid=19, compaction requested=false 2023-02-13 04:56:15,231 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 244360b45c972a590dcc9cdd6803198d: 2023-02-13 04:56:15,231 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,231 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:15,231 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:15,231 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,231 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,231 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,233 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,233 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,233 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,233 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 04:56:15,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@310c9cb1[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:15,234 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 04:56:15,235 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,242 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:15,242 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,242 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:15,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:15,242 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,243 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,243 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:15,244 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,244 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,244 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,244 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,244 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,244 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,249 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,249 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,249 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:15,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:56:15,251 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 04:56:15,251 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:15,251 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:15,251 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,251 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,255 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,255 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,256 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,257 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,257 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,258 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,260 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,260 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,260 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:15,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,260 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:15,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,261 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,361 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:15,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:15,373 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:15,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:15,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,374 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,374 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,374 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 04:56:15,374 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 04:56:15,374 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,374 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,376 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,376 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,376 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,376 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,376 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,376 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:15,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,377 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,377 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,378 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,378 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,378 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,378 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. started... 2023-02-13 04:56:15,378 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:15,379 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,388 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/35ccc4296c59490fafbabb5250d2542f 2023-02-13 04:56:15,397 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/b30508292e5444cea25b5ea9e2da5d66 2023-02-13 04:56:15,400 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/35ccc4296c59490fafbabb5250d2542f as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f 2023-02-13 04:56:15,403 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:15,403 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/b30508292e5444cea25b5ea9e2da5d66 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66 2023-02-13 04:56:15,406 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:15,407 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 29ms, sequenceid=32, compaction requested=true 2023-02-13 04:56:15,407 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 244360b45c972a590dcc9cdd6803198d: 2023-02-13 04:56:15,407 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:15,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:15,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,407 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,409 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,409 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 04:56:15,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@af12233[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:15,411 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 04:56:15,411 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,412 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,413 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:15,412 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,419 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:15,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:56:15,422 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 04:56:15,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:15,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:15,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,422 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:15,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,424 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,425 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,426 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,426 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,426 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,429 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,429 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,429 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,429 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,430 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,530 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:15,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:15,540 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:15,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:15,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,542 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,542 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,542 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 04:56:15,542 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 04:56:15,542 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,542 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,543 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,543 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,543 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,543 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,544 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,544 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,544 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,545 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,545 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,545 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,546 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:15,546 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:15,546 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,546 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. started... 2023-02-13 04:56:15,546 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:15,547 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,548 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,554 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/a2a6920664264825a98c9108240da74a 2023-02-13 04:56:15,566 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/d8cae475caa64cf29b6bace1e98d749d 2023-02-13 04:56:15,570 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/a2a6920664264825a98c9108240da74a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a 2023-02-13 04:56:15,572 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:15,573 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/d8cae475caa64cf29b6bace1e98d749d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d 2023-02-13 04:56:15,576 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:15,576 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 30ms, sequenceid=45, compaction requested=true 2023-02-13 04:56:15,577 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 244360b45c972a590dcc9cdd6803198d: 2023-02-13 04:56:15,577 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:15,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:15,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,578 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,578 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,578 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,578 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 04:56:15,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b2aecda[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:15,580 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 04:56:15,580 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,581 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,581 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:15,581 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,582 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,582 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,582 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,582 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,582 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:15,582 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:15,583 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,583 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,583 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,583 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,585 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,585 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,585 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,585 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,585 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,585 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:15,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:56:15,587 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 04:56:15,587 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:15,587 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:15,587 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,588 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,589 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,592 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,592 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,592 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,593 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,594 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,594 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,604 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,604 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:15,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,605 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,705 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:15,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:15,716 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:15,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:15,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,718 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,718 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,718 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 04:56:15,718 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 04:56:15,718 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,718 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,719 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,719 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,719 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,719 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,719 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,720 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:15,722 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,722 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. started... 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,721 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,722 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,722 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:15,724 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,726 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,730 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/e7ab21dc13284d2e9044e98902142e11 2023-02-13 04:56:15,742 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/e86a01c764ee4901a9d8bf18620c7805 2023-02-13 04:56:15,746 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/e7ab21dc13284d2e9044e98902142e11 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11 2023-02-13 04:56:15,749 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:15,749 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/e86a01c764ee4901a9d8bf18620c7805 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805 2023-02-13 04:56:15,752 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:15,753 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 31ms, sequenceid=58, compaction requested=true 2023-02-13 04:56:15,753 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 244360b45c972a590dcc9cdd6803198d: 2023-02-13 04:56:15,753 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,753 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:15,753 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:15,753 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,753 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,753 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,755 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,755 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,755 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,755 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 04:56:15,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ec03a0d[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:15,757 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 04:56:15,757 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,758 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:15,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:15,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:15,758 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:15,759 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,759 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,759 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,762 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,762 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,762 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:15,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:56:15,765 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 04:56:15,765 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:15,765 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:15,765 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,765 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,767 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,767 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,768 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,768 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,769 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,770 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,770 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,772 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,772 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,773 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,772 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,773 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,774 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,774 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,873 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:15,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:15,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:15,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:15,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,885 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,885 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,885 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 04:56:15,885 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 04:56:15,885 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,885 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,887 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,887 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,887 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,887 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,887 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,887 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 04:56:15,889 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:15,889 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. started... 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,889 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,889 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,890 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,890 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,898 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/1f539ace888a4432b4debb724a5ca1df 2023-02-13 04:56:15,907 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/f32961540c4f40ff9b7a713393075940 2023-02-13 04:56:15,911 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/1f539ace888a4432b4debb724a5ca1df as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df 2023-02-13 04:56:15,914 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:15,914 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/f32961540c4f40ff9b7a713393075940 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940 2023-02-13 04:56:15,917 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:15,918 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 28ms, sequenceid=71, compaction requested=true 2023-02-13 04:56:15,918 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 244360b45c972a590dcc9cdd6803198d: 2023-02-13 04:56:15,918 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 04:56:15,918 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 04:56:15,919 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,919 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,919 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,919 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:15,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 04:56:15,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b7cda70[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:15,921 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 04:56:15,921 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,926 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,926 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,926 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,927 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,927 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:15,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:15,927 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:15,928 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,928 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,928 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,928 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,929 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 04:56:15,929 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,929 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,929 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,929 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,929 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 04:56:15,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:15,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 04:56:15,931 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 04:56:15,931 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:15,931 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:15,931 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,931 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:15,932 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,932 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:15,932 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,932 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,933 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,933 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,933 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:15,935 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,935 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,936 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 04:56:15,939 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,938 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:15,938 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:15,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:15,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:15,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:15,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:15,939 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:15,940 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:15,940 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:15,940 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:15,940 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 04:56:16,039 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:16,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:16,047 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,048 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=244360b45c972a590dcc9cdd6803198d, daughterA=6b649628bf87813fb2490655b7c7b50e, daughterB=52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=244360b45c972a590dcc9cdd6803198d, daughterA=6b649628bf87813fb2490655b7c7b50e, daughterB=52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=244360b45c972a590dcc9cdd6803198d, daughterA=6b649628bf87813fb2490655b7c7b50e, daughterB=52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=244360b45c972a590dcc9cdd6803198d, daughterA=6b649628bf87813fb2490655b7c7b50e, daughterB=52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, UNASSIGN}] 2023-02-13 04:56:16,051 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, UNASSIGN 2023-02-13 04:56:16,051 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=244360b45c972a590dcc9cdd6803198d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,051 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264176051"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264176051"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264176051"}]},"ts":"1676264176051"} 2023-02-13 04:56:16,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 244360b45c972a590dcc9cdd6803198d, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:16,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 244360b45c972a590dcc9cdd6803198d, disabling compactions & flushes 2023-02-13 04:56:16,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:16,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:16,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. after waiting 0 ms 2023-02-13 04:56:16,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:16,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 244360b45c972a590dcc9cdd6803198d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:56:16,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/6ff2ef1b05ee47748a9c942f9bec7634 2023-02-13 04:56:16,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/55f511d21a1f47cf86b2d3c98522ce41 2023-02-13 04:56:16,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf1/6ff2ef1b05ee47748a9c942f9bec7634 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634 2023-02-13 04:56:16,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:16,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.tmp/cf2/55f511d21a1f47cf86b2d3c98522ce41 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41 2023-02-13 04:56:16,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:16,234 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 30ms, sequenceid=82, compaction requested=true 2023-02-13 04:56:16,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:56:16,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:16,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. 2023-02-13 04:56:16,241 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 244360b45c972a590dcc9cdd6803198d: Waiting for close lock at 1676264176204Running coprocessor pre-close hooks at 1676264176204Disabling compacts and flushes for region at 1676264176204Disabling writes for close at 1676264176204Obtaining lock to block concurrent updates at 1676264176204Preparing flush snapshotting stores in 244360b45c972a590dcc9cdd6803198d at 1676264176204Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676264176204Flushing stores of testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. at 1676264176205 (+1 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf1: creating writer at 1676264176207 (+2 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf1: appending metadata at 1676264176208 (+1 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf1: closing flushed file at 1676264176208Flushing 244360b45c972a590dcc9cdd6803198d/cf2: creating writer at 1676264176217 (+9 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf2: appending metadata at 1676264176218 (+1 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf2: closing flushed file at 1676264176219 (+1 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf1: reopening flushed file at 1676264176227 (+8 ms)Flushing 244360b45c972a590dcc9cdd6803198d/cf2: reopening flushed file at 1676264176230 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 244360b45c972a590dcc9cdd6803198d in 30ms, sequenceid=82, compaction requested=true at 1676264176234 (+4 ms)Writing region close event to WAL at 1676264176237 (+3 ms)Running coprocessor post-close hooks at 1676264176240 (+3 ms)Closed at 1676264176240 2023-02-13 04:56:16,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=244360b45c972a590dcc9cdd6803198d, regionState=CLOSED 2023-02-13 04:56:16,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264176242"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264176242"}]},"ts":"1676264176242"} 2023-02-13 04:56:16,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 04:56:16,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 244360b45c972a590dcc9cdd6803198d, server=jenkins-hbase4.apache.org,33859,1676264072829 in 192 msec 2023-02-13 04:56:16,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 04:56:16,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=244360b45c972a590dcc9cdd6803198d, UNASSIGN in 195 msec 2023-02-13 04:56:16,256 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=244360b45c972a590dcc9cdd6803198d, threads=14 2023-02-13 04:56:16,256 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,256 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,256 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,256 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,257 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,257 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,257 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,257 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,258 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,258 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,258 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,258 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,258 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,260 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,262 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849, top=false 2023-02-13 04:56:16,263 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11, top=false 2023-02-13 04:56:16,268 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d, top=false 2023-02-13 04:56:16,269 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940, top=false 2023-02-13 04:56:16,273 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805, top=false 2023-02-13 04:56:16,273 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f, top=false 2023-02-13 04:56:16,274 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66, top=false 2023-02-13 04:56:16,274 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f, top=false 2023-02-13 04:56:16,275 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656, top=false 2023-02-13 04:56:16,276 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:56:16,276 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41, top=false 2023-02-13 04:56:16,277 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800, top=false 2023-02-13 04:56:16,279 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634, top=false 2023-02-13 04:56:16,280 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df, top=false 2023-02-13 04:56:16,289 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,289 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a, top=false 2023-02-13 04:56:16,292 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,297 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,298 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,299 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,300 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,300 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,301 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,301 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,301 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,301 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,304 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,308 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,309 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,309 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,310 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,312 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,313 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,315 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,316 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/.splits/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940 for child: 6b649628bf87813fb2490655b7c7b50e, parent: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,316 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,317 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,317 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,317 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,317 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,318 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,319 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,321 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940 for region: 244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:16,321 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 244360b45c972a590dcc9cdd6803198d Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 04:56:16,353 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:56:16,354 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:56:16,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676264176355"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676264176355"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676264176355"}]},"ts":"1676264176355"} 2023-02-13 04:56:16,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176355"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264176355"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264176355"}]},"ts":"1676264176355"} 2023-02-13 04:56:16,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176355"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264176355"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264176355"}]},"ts":"1676264176355"} 2023-02-13 04:56:16,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, ASSIGN}] 2023-02-13 04:56:16,359 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, ASSIGN 2023-02-13 04:56:16,359 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:16,360 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, ASSIGN 2023-02-13 04:56:16,360 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:16,509 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:56:16,511 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=52b263a0ad5ef658d9c92fe2fd895e55, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,511 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176511"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264176511"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264176511"}]},"ts":"1676264176511"} 2023-02-13 04:56:16,513 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 52b263a0ad5ef658d9c92fe2fd895e55, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:16,515 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6b649628bf87813fb2490655b7c7b50e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,515 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176515"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264176515"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264176515"}]},"ts":"1676264176515"} 2023-02-13 04:56:16,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 6b649628bf87813fb2490655b7c7b50e, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:16,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:16,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6b649628bf87813fb2490655b7c7b50e, NAME => 'testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 04:56:16,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:16,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:16,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,668 INFO [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,669 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1 2023-02-13 04:56:16,669 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1 2023-02-13 04:56:16,669 INFO [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b649628bf87813fb2490655b7c7b50e columnFamilyName cf1 2023-02-13 04:56:16,673 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df 2023-02-13 04:56:16,676 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f 2023-02-13 04:56:16,678 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634 2023-02-13 04:56:16,681 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656 2023-02-13 04:56:16,683 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f 2023-02-13 04:56:16,685 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a 2023-02-13 04:56:16,688 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11 2023-02-13 04:56:16,688 INFO [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(310): Store=6b649628bf87813fb2490655b7c7b50e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:16,688 INFO [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,689 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2 2023-02-13 04:56:16,689 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2 2023-02-13 04:56:16,689 INFO [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b649628bf87813fb2490655b7c7b50e columnFamilyName cf2 2023-02-13 04:56:16,693 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800 2023-02-13 04:56:16,696 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849 2023-02-13 04:56:16,699 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41 2023-02-13 04:56:16,702 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66 2023-02-13 04:56:16,705 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d 2023-02-13 04:56:16,707 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805 2023-02-13 04:56:16,710 DEBUG [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940 2023-02-13 04:56:16,710 INFO [StoreOpener-6b649628bf87813fb2490655b7c7b50e-1] regionserver.HStore(310): Store=6b649628bf87813fb2490655b7c7b50e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:16,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:16,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:16,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6b649628bf87813fb2490655b7c7b50e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65292978, jitterRate=-0.027058809995651245}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:16,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6b649628bf87813fb2490655b7c7b50e: Running coprocessor pre-open hook at 1676264176667Writing region info on filesystem at 1676264176667Initializing all the Stores at 1676264176668 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264176668Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264176668Cleaning up temporary data from old regions at 1676264176712 (+44 ms)Cleaning up detritus from prior splits at 1676264176712Running coprocessor post-open hooks at 1676264176715 (+3 ms)Region opened successfully at 1676264176715 2023-02-13 04:56:16,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e., pid=134, masterSystemTime=1676264176664 2023-02-13 04:56:16,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:16,717 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:16,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:16,717 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:16,717 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 04:56:16,717 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 04:56:16,717 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): 6b649628bf87813fb2490655b7c7b50e/cf1 is initiating minor compaction (all files) 2023-02-13 04:56:16,717 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): 6b649628bf87813fb2490655b7c7b50e/cf2 is initiating minor compaction (all files) 2023-02-13 04:56:16,717 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b649628bf87813fb2490655b7c7b50e/cf1 in testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:16,717 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6b649628bf87813fb2490655b7c7b50e/cf2 in testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:16,717 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/.tmp, totalSize=34.9 K 2023-02-13 04:56:16,717 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/.tmp, totalSize=34.9 K 2023-02-13 04:56:16,718 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264175025 2023-02-13 04:56:16,718 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264175025 2023-02-13 04:56:16,718 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264175189 2023-02-13 04:56:16,718 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264175189 2023-02-13 04:56:16,718 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264175362 2023-02-13 04:56:16,718 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264175362 2023-02-13 04:56:16,719 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264175531 2023-02-13 04:56:16,719 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264175531 2023-02-13 04:56:16,719 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264175707 2023-02-13 04:56:16,719 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264175707 2023-02-13 04:56:16,719 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264175875 2023-02-13 04:56:16,720 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264175875 2023-02-13 04:56:16,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:16,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:16,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:16,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 52b263a0ad5ef658d9c92fe2fd895e55, NAME => 'testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 04:56:16,721 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6b649628bf87813fb2490655b7c7b50e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:16,721 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176721"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264176721"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264176721"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264176721"}]},"ts":"1676264176721"} 2023-02-13 04:56:16,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:16,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-13 04:56:16,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 6b649628bf87813fb2490655b7c7b50e, server=jenkins-hbase4.apache.org,33859,1676264072829 in 207 msec 2023-02-13 04:56:16,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, ASSIGN in 366 msec 2023-02-13 04:56:16,728 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264176041 2023-02-13 04:56:16,728 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264176041 2023-02-13 04:56:16,730 INFO [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,732 DEBUG [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/cf1 2023-02-13 04:56:16,732 DEBUG [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/cf1 2023-02-13 04:56:16,733 INFO [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52b263a0ad5ef658d9c92fe2fd895e55 columnFamilyName cf1 2023-02-13 04:56:16,734 INFO [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] regionserver.HStore(310): Store=52b263a0ad5ef658d9c92fe2fd895e55/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:16,734 INFO [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,735 DEBUG [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/cf2 2023-02-13 04:56:16,735 DEBUG [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/cf2 2023-02-13 04:56:16,735 INFO [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 52b263a0ad5ef658d9c92fe2fd895e55 columnFamilyName cf2 2023-02-13 04:56:16,736 INFO [StoreOpener-52b263a0ad5ef658d9c92fe2fd895e55-1] regionserver.HStore(310): Store=52b263a0ad5ef658d9c92fe2fd895e55/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:16,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,738 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:16,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:16,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 52b263a0ad5ef658d9c92fe2fd895e55; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62703705, jitterRate=-0.06564198434352875}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:16,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 52b263a0ad5ef658d9c92fe2fd895e55: Running coprocessor pre-open hook at 1676264176722Writing region info on filesystem at 1676264176722Initializing all the Stores at 1676264176729 (+7 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264176729Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264176729Cleaning up temporary data from old regions at 1676264176739 (+10 ms)Cleaning up detritus from prior splits at 1676264176739Running coprocessor post-open hooks at 1676264176743 (+4 ms)Region opened successfully at 1676264176743 2023-02-13 04:56:16,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55., pid=133, masterSystemTime=1676264176664 2023-02-13 04:56:16,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:16,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:16,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=52b263a0ad5ef658d9c92fe2fd895e55, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176745"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264176745"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264176745"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264176745"}]},"ts":"1676264176745"} 2023-02-13 04:56:16,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-13 04:56:16,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 52b263a0ad5ef658d9c92fe2fd895e55, server=jenkins-hbase4.apache.org,33859,1676264072829 in 234 msec 2023-02-13 04:56:16,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 04:56:16,750 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, ASSIGN in 390 msec 2023-02-13 04:56:16,750 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6b649628bf87813fb2490655b7c7b50e#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:16,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=244360b45c972a590dcc9cdd6803198d, daughterA=6b649628bf87813fb2490655b7c7b50e, daughterB=52b263a0ad5ef658d9c92fe2fd895e55 in 703 msec 2023-02-13 04:56:16,757 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): 6b649628bf87813fb2490655b7c7b50e#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:16,772 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/.tmp/cf1/d71958f90ebb4f7eb409cd1050ce663a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/d71958f90ebb4f7eb409cd1050ce663a 2023-02-13 04:56:16,774 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/.tmp/cf2/dbd83372b06846f1a12c91f0e0400f6a as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/dbd83372b06846f1a12c91f0e0400f6a 2023-02-13 04:56:16,776 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b649628bf87813fb2490655b7c7b50e/cf1 of 6b649628bf87813fb2490655b7c7b50e into d71958f90ebb4f7eb409cd1050ce663a(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:16,776 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b649628bf87813fb2490655b7c7b50e: 2023-02-13 04:56:16,776 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e., storeName=6b649628bf87813fb2490655b7c7b50e/cf1, priority=9, startTime=1676264176716; duration=0sec 2023-02-13 04:56:16,777 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:16,778 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b649628bf87813fb2490655b7c7b50e/cf2 of 6b649628bf87813fb2490655b7c7b50e into dbd83372b06846f1a12c91f0e0400f6a(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:16,778 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6b649628bf87813fb2490655b7c7b50e: 2023-02-13 04:56:16,778 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e., storeName=6b649628bf87813fb2490655b7c7b50e/cf2, priority=9, startTime=1676264176717; duration=0sec 2023-02-13 04:56:16,779 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:16,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33859] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:50508 deadline: 1676264236799, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d. is not online on jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,908 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 04:56:16,911 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:16,912 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 04:56:16,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-13 04:56:16,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:16,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 04:56:16,915 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264176915"}]},"ts":"1676264176915"} 2023-02-13 04:56:16,916 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 04:56:16,917 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 04:56:16,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, UNASSIGN}] 2023-02-13 04:56:16,919 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, UNASSIGN 2023-02-13 04:56:16,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=52b263a0ad5ef658d9c92fe2fd895e55, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176919"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264176919"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264176919"}]},"ts":"1676264176919"} 2023-02-13 04:56:16,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 52b263a0ad5ef658d9c92fe2fd895e55, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:16,922 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, UNASSIGN 2023-02-13 04:56:16,923 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6b649628bf87813fb2490655b7c7b50e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:16,923 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264176923"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264176923"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264176923"}]},"ts":"1676264176923"} 2023-02-13 04:56:16,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 6b649628bf87813fb2490655b7c7b50e, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:17,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 04:56:17,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:17,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 52b263a0ad5ef658d9c92fe2fd895e55, disabling compactions & flushes 2023-02-13 04:56:17,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:17,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:17,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. after waiting 0 ms 2023-02-13 04:56:17,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:17,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 04:56:17,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:17,076 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55. 2023-02-13 04:56:17,076 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 52b263a0ad5ef658d9c92fe2fd895e55: Waiting for close lock at 1676264177072Running coprocessor pre-close hooks at 1676264177072Disabling compacts and flushes for region at 1676264177072Disabling writes for close at 1676264177073 (+1 ms)Writing region close event to WAL at 1676264177073Running coprocessor post-close hooks at 1676264177076 (+3 ms)Closed at 1676264177076 2023-02-13 04:56:17,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:17,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:17,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6b649628bf87813fb2490655b7c7b50e, disabling compactions & flushes 2023-02-13 04:56:17,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:17,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:17,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. after waiting 0 ms 2023-02-13 04:56:17,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:17,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=52b263a0ad5ef658d9c92fe2fd895e55, regionState=CLOSED 2023-02-13 04:56:17,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264177078"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264177078"}]},"ts":"1676264177078"} 2023-02-13 04:56:17,078 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634] to archive 2023-02-13 04:56:17,079 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:17,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 04:56:17,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 52b263a0ad5ef658d9c92fe2fd895e55, server=jenkins-hbase4.apache.org,33859,1676264072829 in 159 msec 2023-02-13 04:56:17,080 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-78c5c6a80efe4b3380352496b657a656 2023-02-13 04:56:17,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=52b263a0ad5ef658d9c92fe2fd895e55, UNASSIGN in 162 msec 2023-02-13 04:56:17,082 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-88bbfde6097b47b1a218117edc2de36f 2023-02-13 04:56:17,083 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-35ccc4296c59490fafbabb5250d2542f 2023-02-13 04:56:17,084 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-a2a6920664264825a98c9108240da74a 2023-02-13 04:56:17,085 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e7ab21dc13284d2e9044e98902142e11 2023-02-13 04:56:17,086 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-1f539ace888a4432b4debb724a5ca1df 2023-02-13 04:56:17,087 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-6ff2ef1b05ee47748a9c942f9bec7634 2023-02-13 04:56:17,088 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41] to archive 2023-02-13 04:56:17,088 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:17,090 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-4f6cdedaebbf486a84bd578702fb6800 2023-02-13 04:56:17,091 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-548d76689eb94fb687138e6c7a80d849 2023-02-13 04:56:17,091 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-b30508292e5444cea25b5ea9e2da5d66 2023-02-13 04:56:17,092 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-d8cae475caa64cf29b6bace1e98d749d 2023-02-13 04:56:17,093 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-e86a01c764ee4901a9d8bf18620c7805 2023-02-13 04:56:17,094 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-f32961540c4f40ff9b7a713393075940 2023-02-13 04:56:17,095 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/testSplitTableRegionEmptyDaughter=244360b45c972a590dcc9cdd6803198d-55f511d21a1f47cf86b2d3c98522ce41 2023-02-13 04:56:17,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:56:17,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:17,098 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e. 2023-02-13 04:56:17,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6b649628bf87813fb2490655b7c7b50e: Waiting for close lock at 1676264177078Running coprocessor pre-close hooks at 1676264177078Disabling compacts and flushes for region at 1676264177078Disabling writes for close at 1676264177078Writing region close event to WAL at 1676264177096 (+18 ms)Running coprocessor post-close hooks at 1676264177098 (+2 ms)Closed at 1676264177098 2023-02-13 04:56:17,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:17,100 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6b649628bf87813fb2490655b7c7b50e, regionState=CLOSED 2023-02-13 04:56:17,100 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676264177100"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264177100"}]},"ts":"1676264177100"} 2023-02-13 04:56:17,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 04:56:17,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 6b649628bf87813fb2490655b7c7b50e, server=jenkins-hbase4.apache.org,33859,1676264072829 in 177 msec 2023-02-13 04:56:17,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-13 04:56:17,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6b649628bf87813fb2490655b7c7b50e, UNASSIGN in 184 msec 2023-02-13 04:56:17,104 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264177104"}]},"ts":"1676264177104"} 2023-02-13 04:56:17,104 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 04:56:17,106 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 04:56:17,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 193 msec 2023-02-13 04:56:17,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 04:56:17,217 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 04:56:17,218 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,221 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 04:56:17,224 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:17,224 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:17,224 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:17,226 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/recovered.edits] 2023-02-13 04:56:17,227 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/recovered.edits] 2023-02-13 04:56:17,227 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/recovered.edits] 2023-02-13 04:56:17,230 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/d71958f90ebb4f7eb409cd1050ce663a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf1/d71958f90ebb4f7eb409cd1050ce663a 2023-02-13 04:56:17,230 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/1f539ace888a4432b4debb724a5ca1df 2023-02-13 04:56:17,231 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/35ccc4296c59490fafbabb5250d2542f 2023-02-13 04:56:17,232 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/dbd83372b06846f1a12c91f0e0400f6a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/cf2/dbd83372b06846f1a12c91f0e0400f6a 2023-02-13 04:56:17,232 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/recovered.edits/88.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55/recovered.edits/88.seqid 2023-02-13 04:56:17,233 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/6ff2ef1b05ee47748a9c942f9bec7634 2023-02-13 04:56:17,233 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/52b263a0ad5ef658d9c92fe2fd895e55 2023-02-13 04:56:17,234 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/78c5c6a80efe4b3380352496b657a656 2023-02-13 04:56:17,235 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e/recovered.edits/90.seqid 2023-02-13 04:56:17,236 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/6b649628bf87813fb2490655b7c7b50e 2023-02-13 04:56:17,236 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/88bbfde6097b47b1a218117edc2de36f 2023-02-13 04:56:17,237 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/a2a6920664264825a98c9108240da74a 2023-02-13 04:56:17,238 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf1/e7ab21dc13284d2e9044e98902142e11 2023-02-13 04:56:17,241 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/4f6cdedaebbf486a84bd578702fb6800 2023-02-13 04:56:17,242 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/548d76689eb94fb687138e6c7a80d849 2023-02-13 04:56:17,243 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/55f511d21a1f47cf86b2d3c98522ce41 2023-02-13 04:56:17,245 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/b30508292e5444cea25b5ea9e2da5d66 2023-02-13 04:56:17,246 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/d8cae475caa64cf29b6bace1e98d749d 2023-02-13 04:56:17,247 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/e86a01c764ee4901a9d8bf18620c7805 2023-02-13 04:56:17,248 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/cf2/f32961540c4f40ff9b7a713393075940 2023-02-13 04:56:17,250 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/recovered.edits/85.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d/recovered.edits/85.seqid 2023-02-13 04:56:17,250 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegionEmptyDaughter/244360b45c972a590dcc9cdd6803198d 2023-02-13 04:56:17,250 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 04:56:17,252 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,254 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 04:56:17,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 04:56:17,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 04:56:17,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264177256"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:17,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264177256"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:17,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264177256"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:17,258 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:56:17,258 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 244360b45c972a590dcc9cdd6803198d, NAME => 'testSplitTableRegionEmptyDaughter,,1676264174521.244360b45c972a590dcc9cdd6803198d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6b649628bf87813fb2490655b7c7b50e, NAME => 'testSplitTableRegionEmptyDaughter,,1676264176047.6b649628bf87813fb2490655b7c7b50e.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 52b263a0ad5ef658d9c92fe2fd895e55, NAME => 'testSplitTableRegionEmptyDaughter,71,1676264176047.52b263a0ad5ef658d9c92fe2fd895e55.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 04:56:17,258 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 04:56:17,258 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264177258"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:17,259 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 04:56:17,261 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 04:56:17,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 43 msec 2023-02-13 04:56:17,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 04:56:17,322 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 04:56:17,336 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=841 (was 843), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=114 (was 117), ProcessCount=168 (was 168), AvailableMemoryMB=3478 (was 3482) 2023-02-13 04:56:17,349 INFO [Listener at localhost/33551] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=114, ProcessCount=168, AvailableMemoryMB=3478 2023-02-13 04:56:17,349 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:17,352 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 04:56:17,353 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 04:56:17,354 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 04:56:17,355 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 04:56:17,356 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 04:56:17,357 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,357 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e empty. 2023-02-13 04:56:17,358 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,358 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 04:56:17,367 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 04:56:17,368 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 714bc3f9fc4c0faad6c7966de7ffb94e, NAME => 'testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp 2023-02-13 04:56:17,374 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:17,374 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 714bc3f9fc4c0faad6c7966de7ffb94e, disabling compactions & flushes 2023-02-13 04:56:17,374 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,375 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,375 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. after waiting 0 ms 2023-02-13 04:56:17,375 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,375 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,375 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 714bc3f9fc4c0faad6c7966de7ffb94e: Waiting for close lock at 1676264177374Disabling compacts and flushes for region at 1676264177374Disabling writes for close at 1676264177375 (+1 ms)Writing region close event to WAL at 1676264177375Closed at 1676264177375 2023-02-13 04:56:17,376 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 04:56:17,377 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264177377"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264177377"}]},"ts":"1676264177377"} 2023-02-13 04:56:17,378 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 04:56:17,379 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 04:56:17,379 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264177379"}]},"ts":"1676264177379"} 2023-02-13 04:56:17,380 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 04:56:17,382 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 04:56:17,383 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 04:56:17,383 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 04:56:17,383 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 04:56:17,383 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 04:56:17,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, ASSIGN}] 2023-02-13 04:56:17,384 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, ASSIGN 2023-02-13 04:56:17,385 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:17,535 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 04:56:17,536 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=714bc3f9fc4c0faad6c7966de7ffb94e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,536 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264177536"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264177536"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264177536"}]},"ts":"1676264177536"} 2023-02-13 04:56:17,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 714bc3f9fc4c0faad6c7966de7ffb94e, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:17,539 ERROR [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid 82110bba-6b14-4557-aa3b-b536cbd539dd) service to localhost/127.0.0.1:44231 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:56:17,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 714bc3f9fc4c0faad6c7966de7ffb94e, NAME => 'testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 04:56:17,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:17,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:17,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,694 INFO [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,695 DEBUG [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1 2023-02-13 04:56:17,696 DEBUG [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1 2023-02-13 04:56:17,696 INFO [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 714bc3f9fc4c0faad6c7966de7ffb94e columnFamilyName cf1 2023-02-13 04:56:17,697 INFO [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] regionserver.HStore(310): Store=714bc3f9fc4c0faad6c7966de7ffb94e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:17,697 INFO [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,698 DEBUG [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2 2023-02-13 04:56:17,698 DEBUG [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2 2023-02-13 04:56:17,698 INFO [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 714bc3f9fc4c0faad6c7966de7ffb94e columnFamilyName cf2 2023-02-13 04:56:17,699 INFO [StoreOpener-714bc3f9fc4c0faad6c7966de7ffb94e-1] regionserver.HStore(310): Store=714bc3f9fc4c0faad6c7966de7ffb94e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:17,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:17,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:17,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 04:56:17,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 714bc3f9fc4c0faad6c7966de7ffb94e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73793010, jitterRate=0.09960153698921204}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:17,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 714bc3f9fc4c0faad6c7966de7ffb94e: Running coprocessor pre-open hook at 1676264177692Writing region info on filesystem at 1676264177692Initializing all the Stores at 1676264177693 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264177693Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264177693Cleaning up temporary data from old regions at 1676264177701 (+8 ms)Cleaning up detritus from prior splits at 1676264177701Running coprocessor post-open hooks at 1676264177705 (+4 ms)Region opened successfully at 1676264177705 2023-02-13 04:56:17,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e., pid=143, masterSystemTime=1676264177689 2023-02-13 04:56:17,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=714bc3f9fc4c0faad6c7966de7ffb94e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,708 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264177707"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264177707"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264177707"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264177707"}]},"ts":"1676264177707"} 2023-02-13 04:56:17,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 04:56:17,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 714bc3f9fc4c0faad6c7966de7ffb94e, server=jenkins-hbase4.apache.org,33859,1676264072829 in 172 msec 2023-02-13 04:56:17,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 04:56:17,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, ASSIGN in 327 msec 2023-02-13 04:56:17,712 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 04:56:17,712 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264177712"}]},"ts":"1676264177712"} 2023-02-13 04:56:17,713 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 04:56:17,715 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 04:56:17,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 362 msec 2023-02-13 04:56:17,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:17,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 04:56:17,862 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 04:56:17,862 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:17,862 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 04:56:17,862 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 04:56:17,862 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,862 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,864 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:17,864 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:17,864 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:17,864 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:17,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:17,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:17,864 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,864 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:17,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:17,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:17,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,866 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:17,866 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:17,866 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:17,866 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:17,866 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:17,866 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,866 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. started... 2023-02-13 04:56:17,867 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 04:56:17,868 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,868 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,868 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,868 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:17,869 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,869 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:17,878 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/0a2017bd3763487981d2d4a944afe414 2023-02-13 04:56:17,891 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/9fa45b953da34be4848951d53fd0b71e 2023-02-13 04:56:17,894 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/0a2017bd3763487981d2d4a944afe414 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414 2023-02-13 04:56:17,897 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:17,898 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/9fa45b953da34be4848951d53fd0b71e as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e 2023-02-13 04:56:17,900 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:17,901 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 34ms, sequenceid=6, compaction requested=false 2023-02-13 04:56:17,901 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 04:56:17,901 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 714bc3f9fc4c0faad6c7966de7ffb94e: 2023-02-13 04:56:17,901 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:17,902 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:17,902 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:17,902 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,902 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:17,902 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:17,903 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,903 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:17,903 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,903 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:17,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:17,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 04:56:17,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@381f9796[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:17,905 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 04:56:17,905 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:17,908 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:17,908 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:17,907 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,908 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:17,908 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:17,907 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:17,909 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:17,909 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:17,909 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,909 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:17,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:17,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:17,913 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:17,913 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:17,913 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:17,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:17,913 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:17,913 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:17,913 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:17,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:17,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:17,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:17,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:17,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 04:56:17,915 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 04:56:17,916 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:17,916 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:17,916 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 04:56:17,916 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:17,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,918 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:17,918 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:17,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:17,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:17,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:17,919 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:17,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:17,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:17,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:17,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,920 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:17,921 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:17,921 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:17,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:17,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,923 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 04:56:17,923 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:17,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:17,923 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:17,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:17,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:17,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:17,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:17,923 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:17,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:17,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:17,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:17,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:17,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:17,924 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,024 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:18,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:18,035 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:18,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:18,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 04:56:18,036 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 04:56:18,036 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,036 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 04:56:18,037 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 04:56:18,037 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,037 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,038 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,038 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,038 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,038 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,038 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,038 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,040 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,041 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:18,041 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,041 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. started... 2023-02-13 04:56:18,041 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,057 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/799becb3b2904cb4be3815a38154facb 2023-02-13 04:56:18,067 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/1eb1baa91f964285900c7cde94bae435 2023-02-13 04:56:18,070 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/799becb3b2904cb4be3815a38154facb as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb 2023-02-13 04:56:18,073 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:18,074 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/1eb1baa91f964285900c7cde94bae435 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435 2023-02-13 04:56:18,077 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 04:56:18,078 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 37ms, sequenceid=19, compaction requested=false 2023-02-13 04:56:18,078 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 714bc3f9fc4c0faad6c7966de7ffb94e: 2023-02-13 04:56:18,078 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,078 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:18,078 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:18,078 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,078 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,078 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,079 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,079 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,080 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,080 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 04:56:18,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14ffe073[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:18,081 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 04:56:18,081 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,082 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,083 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:18,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,083 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:18,083 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:18,084 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,084 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,084 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,088 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:18,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 04:56:18,091 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 04:56:18,091 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:18,091 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:18,091 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 04:56:18,091 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,092 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,093 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,093 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,095 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,095 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,095 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,097 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 04:56:18,097 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,097 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,097 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,098 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,198 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:18,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:18,210 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:18,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:18,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 04:56:18,211 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 04:56:18,211 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,211 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 04:56:18,211 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 04:56:18,212 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,212 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,213 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,213 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,213 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,213 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:18,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,214 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,214 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,215 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,215 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,217 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,217 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. started... 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,217 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,217 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,221 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,221 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,221 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,227 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/43801071b8bf40599c613a1d39886224 2023-02-13 04:56:18,236 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/ec7b80b6c87e4947a6729118372c53ba 2023-02-13 04:56:18,239 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/43801071b8bf40599c613a1d39886224 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224 2023-02-13 04:56:18,242 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:18,243 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/ec7b80b6c87e4947a6729118372c53ba as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba 2023-02-13 04:56:18,246 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 04:56:18,246 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 29ms, sequenceid=32, compaction requested=true 2023-02-13 04:56:18,247 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 714bc3f9fc4c0faad6c7966de7ffb94e: 2023-02-13 04:56:18,247 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,247 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,248 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,248 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,248 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,248 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 04:56:18,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b00b0b3[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:18,250 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 04:56:18,250 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,252 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:18,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,252 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,253 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:18,254 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,254 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,254 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,254 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,255 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,255 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,255 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,255 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,255 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,255 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:18,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 04:56:18,257 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 04:56:18,257 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:18,257 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:18,257 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 04:56:18,257 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:18,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,258 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,259 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,259 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,259 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,260 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,261 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,262 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,262 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,268 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,268 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,268 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,268 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,268 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,268 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,269 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 04:56:18,269 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,269 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:18,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:18,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,269 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,270 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,270 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,270 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,270 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,270 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,370 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:18,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:18,380 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:18,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:18,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 04:56:18,381 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 04:56:18,381 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,381 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 04:56:18,381 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 04:56:18,382 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,382 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,383 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,383 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,383 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,383 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,384 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,385 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,385 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,385 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,385 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,385 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,385 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:18,385 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:18,385 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,385 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. started... 2023-02-13 04:56:18,385 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:18,386 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,386 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,387 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,387 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,387 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,387 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,394 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/f06f89a77ad7447587594280ad7dc453 2023-02-13 04:56:18,409 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/4db7c7a4ca794d18888df2951db6b353 2023-02-13 04:56:18,412 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/f06f89a77ad7447587594280ad7dc453 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453 2023-02-13 04:56:18,415 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:18,415 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/4db7c7a4ca794d18888df2951db6b353 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353 2023-02-13 04:56:18,418 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 04:56:18,419 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 34ms, sequenceid=45, compaction requested=true 2023-02-13 04:56:18,419 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 714bc3f9fc4c0faad6c7966de7ffb94e: 2023-02-13 04:56:18,419 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:18,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:18,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,419 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,420 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,420 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,421 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,421 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 04:56:18,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26f24af9[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:18,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 04:56:18,422 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,423 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,424 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,424 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:18,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:18,424 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,425 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,427 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,427 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,427 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:18,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 04:56:18,430 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 04:56:18,431 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:18,431 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:18,431 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 04:56:18,431 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:18,432 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,432 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,433 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,432 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,433 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,434 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,434 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,436 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,436 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,436 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,446 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 04:56:18,446 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:18,446 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,446 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:18,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,447 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,526 ERROR [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid 2af29379-a15b-4462-998d-be0242ce1402) service to localhost/127.0.0.1:44231 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 04:56:18,547 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:18,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:18,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:18,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:18,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 04:56:18,560 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 04:56:18,560 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,560 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 04:56:18,560 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 04:56:18,560 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,560 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,562 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,562 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,562 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,562 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,563 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,564 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,564 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,565 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:18,565 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:18,565 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,565 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. started... 2023-02-13 04:56:18,565 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,566 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,574 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/887a50d0c36142e6925ea0357a455816 2023-02-13 04:56:18,584 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/de7538ae9d62408c8d38e5953bbd6d6d 2023-02-13 04:56:18,587 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/887a50d0c36142e6925ea0357a455816 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816 2023-02-13 04:56:18,589 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:18,590 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/de7538ae9d62408c8d38e5953bbd6d6d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d 2023-02-13 04:56:18,593 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 04:56:18,593 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 28ms, sequenceid=58, compaction requested=true 2023-02-13 04:56:18,594 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 714bc3f9fc4c0faad6c7966de7ffb94e: 2023-02-13 04:56:18,594 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,594 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:18,594 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:18,594 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,594 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,594 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,595 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,595 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,595 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,595 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 04:56:18,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6295ca96[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:18,597 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 04:56:18,597 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:18,599 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,599 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,599 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:18,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,600 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,600 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:18,600 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,600 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:18,600 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:18,600 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:18,601 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,601 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,601 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,602 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,602 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,602 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,602 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:18,602 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,602 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:18,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 04:56:18,605 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 04:56:18,605 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:18,605 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:18,605 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 04:56:18,605 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:18,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,607 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,607 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:18,608 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:18,610 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:18,610 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,611 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,615 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,616 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:18,616 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 04:56:18,616 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:18,616 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:18,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,616 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,616 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:18,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,617 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:18,617 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:18,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:18,617 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:18,617 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:18,617 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,617 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,717 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:18,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:18,728 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 04:56:18,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 04:56:18,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 04:56:18,730 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 04:56:18,730 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,730 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 04:56:18,730 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 04:56:18,730 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,730 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,731 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,731 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,731 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,731 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:18,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,731 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:18,731 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 04:56:18,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:18,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:18,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 04:56:18,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 04:56:18,733 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,733 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36009,1676264072859' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46601,1676264072787' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 04:56:18,734 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 04:56:18,734 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:18,734 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. started... 2023-02-13 04:56:18,734 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 04:56:18,735 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,735 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,735 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,735 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:18,736 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:18,736 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:19,146 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/650ed5d5a6cc4516ab9acdeebb47d04c 2023-02-13 04:56:19,556 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/95e62fee42144a65b14141c5f3425167 2023-02-13 04:56:19,559 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/650ed5d5a6cc4516ab9acdeebb47d04c as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c 2023-02-13 04:56:19,563 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:19,564 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/95e62fee42144a65b14141c5f3425167 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167 2023-02-13 04:56:19,568 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 04:56:19,569 INFO [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 835ms, sequenceid=71, compaction requested=true 2023-02-13 04:56:19,569 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 714bc3f9fc4c0faad6c7966de7ffb94e: 2023-02-13 04:56:19,569 DEBUG [rs(jenkins-hbase4.apache.org,33859,1676264072829)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:19,569 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 04:56:19,569 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 04:56:19,569 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,569 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 04:56:19,569 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 04:56:19,572 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,572 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:19,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:19,572 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,572 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 04:56:19,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:19,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:19,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:19,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:19,574 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33859,1676264072829' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 04:56:19,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@90252cf[Count = 0] remaining members to acquire global barrier 2023-02-13 04:56:19,575 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 04:56:19,575 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:19,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,576 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 04:56:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:19,577 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36009,1676264072859' in zk 2023-02-13 04:56:19,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,576 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:19,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:19,577 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33859,1676264072829' in zk 2023-02-13 04:56:19,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,577 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 04:56:19,577 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 04:56:19,577 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46601,1676264072787' in zk 2023-02-13 04:56:19,584 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:19,584 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,584 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:19,584 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:19,584 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:19,584 DEBUG [member: 'jenkins-hbase4.apache.org,36009,1676264072859' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:19,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:19,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:19,584 DEBUG [member: 'jenkins-hbase4.apache.org,33859,1676264072829' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 04:56:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:19,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,46601,1676264072787' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 04:56:19,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:19,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:19,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:19,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:19,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33859,1676264072829': 2023-02-13 04:56:19,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33859,1676264072829' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 04:56:19,588 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 04:56:19,588 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 04:56:19,588 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 04:56:19,588 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 04:56:19,588 INFO [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 04:56:19,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:19,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,590 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:19,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 04:56:19,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:19,591 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:19,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,591 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:19,592 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 04:56:19,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:19,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,594 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 04:56:19,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 04:56:19,594 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,595 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,600 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,600 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:19,600 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:19,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:19,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:19,600 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 04:56:19,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:19,601 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 04:56:19,601 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 04:56:19,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:19,600 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 04:56:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:19,601 DEBUG [(jenkins-hbase4.apache.org,43801,1676264071669)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 04:56:19,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 04:56:19,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 04:56:19,601 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:19,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:19,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 04:56:19,602 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 04:56:19,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:19,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:19,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 04:56:19,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 04:56:19,702 DEBUG [Listener at localhost/33551] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 04:56:19,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 04:56:19,708 DEBUG [Listener at localhost/33551] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,709 DEBUG [Listener at localhost/33551] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=714bc3f9fc4c0faad6c7966de7ffb94e, daughterA=3178d3aa6e4f4233a6aa9320e64e5156, daughterB=456807945687418d7719c5317d1c58d1 2023-02-13 04:56:19,710 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=714bc3f9fc4c0faad6c7966de7ffb94e, daughterA=3178d3aa6e4f4233a6aa9320e64e5156, daughterB=456807945687418d7719c5317d1c58d1 2023-02-13 04:56:19,710 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=714bc3f9fc4c0faad6c7966de7ffb94e, daughterA=3178d3aa6e4f4233a6aa9320e64e5156, daughterB=456807945687418d7719c5317d1c58d1 2023-02-13 04:56:19,710 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=714bc3f9fc4c0faad6c7966de7ffb94e, daughterA=3178d3aa6e4f4233a6aa9320e64e5156, daughterB=456807945687418d7719c5317d1c58d1 2023-02-13 04:56:19,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, UNASSIGN}] 2023-02-13 04:56:19,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, UNASSIGN 2023-02-13 04:56:19,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=714bc3f9fc4c0faad6c7966de7ffb94e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:19,712 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264179712"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264179712"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264179712"}]},"ts":"1676264179712"} 2023-02-13 04:56:19,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 714bc3f9fc4c0faad6c7966de7ffb94e, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:19,865 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 714bc3f9fc4c0faad6c7966de7ffb94e, disabling compactions & flushes 2023-02-13 04:56:19,865 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:19,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:19,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. after waiting 0 ms 2023-02-13 04:56:19,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:19,865 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 714bc3f9fc4c0faad6c7966de7ffb94e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 04:56:19,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/06cfff59c698402299e9e01846cee652 2023-02-13 04:56:19,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/31a8f2b8e62d4221bf44b3c32f4314ba 2023-02-13 04:56:19,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf1/06cfff59c698402299e9e01846cee652 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652 2023-02-13 04:56:19,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:19,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/.tmp/cf2/31a8f2b8e62d4221bf44b3c32f4314ba as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba 2023-02-13 04:56:19,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 04:56:19,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 31ms, sequenceid=82, compaction requested=true 2023-02-13 04:56:19,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 04:56:19,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:19,901 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. 2023-02-13 04:56:19,901 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 714bc3f9fc4c0faad6c7966de7ffb94e: Waiting for close lock at 1676264179865Running coprocessor pre-close hooks at 1676264179865Disabling compacts and flushes for region at 1676264179865Disabling writes for close at 1676264179865Obtaining lock to block concurrent updates at 1676264179865Preparing flush snapshotting stores in 714bc3f9fc4c0faad6c7966de7ffb94e at 1676264179865Finished memstore snapshotting testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676264179866 (+1 ms)Flushing stores of testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. at 1676264179866Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf1: creating writer at 1676264179867 (+1 ms)Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf1: appending metadata at 1676264179869 (+2 ms)Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf1: closing flushed file at 1676264179869Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf2: creating writer at 1676264179877 (+8 ms)Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf2: appending metadata at 1676264179879 (+2 ms)Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf2: closing flushed file at 1676264179879Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf1: reopening flushed file at 1676264179889 (+10 ms)Flushing 714bc3f9fc4c0faad6c7966de7ffb94e/cf2: reopening flushed file at 1676264179893 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 714bc3f9fc4c0faad6c7966de7ffb94e in 31ms, sequenceid=82, compaction requested=true at 1676264179896 (+3 ms)Writing region close event to WAL at 1676264179898 (+2 ms)Running coprocessor post-close hooks at 1676264179900 (+2 ms)Closed at 1676264179901 (+1 ms) 2023-02-13 04:56:19,902 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,902 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=714bc3f9fc4c0faad6c7966de7ffb94e, regionState=CLOSED 2023-02-13 04:56:19,903 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264179902"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264179902"}]},"ts":"1676264179902"} 2023-02-13 04:56:19,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 04:56:19,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 714bc3f9fc4c0faad6c7966de7ffb94e, server=jenkins-hbase4.apache.org,33859,1676264072829 in 191 msec 2023-02-13 04:56:19,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 04:56:19,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=714bc3f9fc4c0faad6c7966de7ffb94e, UNASSIGN in 194 msec 2023-02-13 04:56:19,915 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=714bc3f9fc4c0faad6c7966de7ffb94e, threads=14 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,915 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,916 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,916 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,920 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,920 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,920 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:19,920 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,100 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,108 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,121 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,128 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,134 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,140 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,142 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,143 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,417 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,465 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,492 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453 for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,513 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,890 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,907 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d for region: 714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:20,907 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 714bc3f9fc4c0faad6c7966de7ffb94e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 04:56:20,941 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:56:20,943 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 04:56:20,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676264180944"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676264180944"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676264180944"}]},"ts":"1676264180944"} 2023-02-13 04:56:20,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264180944"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264180944"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264180944"}]},"ts":"1676264180944"} 2023-02-13 04:56:20,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264180944"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264180944"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264180944"}]},"ts":"1676264180944"} 2023-02-13 04:56:20,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, ASSIGN}] 2023-02-13 04:56:20,948 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, ASSIGN 2023-02-13 04:56:20,948 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:20,949 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, ASSIGN 2023-02-13 04:56:20,949 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33859,1676264072829; forceNewPlan=false, retain=false 2023-02-13 04:56:21,099 INFO [jenkins-hbase4:43801] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 04:56:21,100 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=456807945687418d7719c5317d1c58d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,100 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181100"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264181100"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264181100"}]},"ts":"1676264181100"} 2023-02-13 04:56:21,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 456807945687418d7719c5317d1c58d1, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:21,104 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=3178d3aa6e4f4233a6aa9320e64e5156, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,104 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181104"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264181104"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264181104"}]},"ts":"1676264181104"} 2023-02-13 04:56:21,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure 3178d3aa6e4f4233a6aa9320e64e5156, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:21,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3178d3aa6e4f4233a6aa9320e64e5156, NAME => 'testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 04:56:21,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:21,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:21,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,258 INFO [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,258 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1 2023-02-13 04:56:21,259 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1 2023-02-13 04:56:21,259 INFO [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3178d3aa6e4f4233a6aa9320e64e5156 columnFamilyName cf1 2023-02-13 04:56:21,270 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652-bottom 2023-02-13 04:56:21,274 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414-bottom 2023-02-13 04:56:21,277 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224-bottom 2023-02-13 04:56:21,280 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c-bottom 2023-02-13 04:56:21,284 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb-bottom 2023-02-13 04:56:21,287 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816-bottom 2023-02-13 04:56:21,291 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453-bottom 2023-02-13 04:56:21,291 INFO [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(310): Store=3178d3aa6e4f4233a6aa9320e64e5156/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:21,291 INFO [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,292 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2 2023-02-13 04:56:21,292 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2 2023-02-13 04:56:21,292 INFO [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3178d3aa6e4f4233a6aa9320e64e5156 columnFamilyName cf2 2023-02-13 04:56:21,303 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435-bottom 2023-02-13 04:56:21,306 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba-bottom 2023-02-13 04:56:21,309 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353-bottom 2023-02-13 04:56:21,312 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167-bottom 2023-02-13 04:56:21,314 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e-bottom 2023-02-13 04:56:21,318 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d-bottom 2023-02-13 04:56:21,321 DEBUG [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba-bottom 2023-02-13 04:56:21,321 INFO [StoreOpener-3178d3aa6e4f4233a6aa9320e64e5156-1] regionserver.HStore(310): Store=3178d3aa6e4f4233a6aa9320e64e5156/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:21,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:21,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3178d3aa6e4f4233a6aa9320e64e5156; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60172440, jitterRate=-0.10336077213287354}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:21,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3178d3aa6e4f4233a6aa9320e64e5156: Running coprocessor pre-open hook at 1676264181257Writing region info on filesystem at 1676264181257Initializing all the Stores at 1676264181257Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264181258 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264181258Cleaning up temporary data from old regions at 1676264181323 (+65 ms)Cleaning up detritus from prior splits at 1676264181323Running coprocessor post-open hooks at 1676264181325 (+2 ms)Region opened successfully at 1676264181325 2023-02-13 04:56:21,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156., pid=150, masterSystemTime=1676264181253 2023-02-13 04:56:21,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 04:56:21,326 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:21,326 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:21,326 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,326 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): 3178d3aa6e4f4233a6aa9320e64e5156/cf1 is initiating minor compaction (all files) 2023-02-13 04:56:21,326 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,326 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3178d3aa6e4f4233a6aa9320e64e5156/cf1 in testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,327 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): 3178d3aa6e4f4233a6aa9320e64e5156/cf2 is initiating minor compaction (all files) 2023-02-13 04:56:21,327 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3178d3aa6e4f4233a6aa9320e64e5156/cf2 in testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,327 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/.tmp, totalSize=34.9 K 2023-02-13 04:56:21,327 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba-bottom] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/.tmp, totalSize=34.9 K 2023-02-13 04:56:21,327 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264177856 2023-02-13 04:56:21,327 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676264177856 2023-02-13 04:56:21,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,328 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264178026 2023-02-13 04:56:21,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,328 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676264178026 2023-02-13 04:56:21,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 456807945687418d7719c5317d1c58d1, NAME => 'testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 04:56:21,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=3178d3aa6e4f4233a6aa9320e64e5156, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,328 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181328"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264181328"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264181328"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264181328"}]},"ts":"1676264181328"} 2023-02-13 04:56:21,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 04:56:21,328 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264178200 2023-02-13 04:56:21,328 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676264178200 2023-02-13 04:56:21,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 04:56:21,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,329 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,329 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264178371 2023-02-13 04:56:21,329 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676264178371 2023-02-13 04:56:21,329 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264178549 2023-02-13 04:56:21,329 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676264178549 2023-02-13 04:56:21,329 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264178718 2023-02-13 04:56:21,330 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676264178718 2023-02-13 04:56:21,330 INFO [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,330 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264179703 2023-02-13 04:56:21,330 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676264179703 2023-02-13 04:56:21,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-13 04:56:21,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure 3178d3aa6e4f4233a6aa9320e64e5156, server=jenkins-hbase4.apache.org,33859,1676264072829 in 224 msec 2023-02-13 04:56:21,332 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1 2023-02-13 04:56:21,332 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1 2023-02-13 04:56:21,332 INFO [StoreOpener-456807945687418d7719c5317d1c58d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 456807945687418d7719c5317d1c58d1 columnFamilyName cf1 2023-02-13 04:56:21,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, ASSIGN in 385 msec 2023-02-13 04:56:21,340 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): 3178d3aa6e4f4233a6aa9320e64e5156#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:21,341 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3178d3aa6e4f4233a6aa9320e64e5156#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:21,344 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652-top 2023-02-13 04:56:21,353 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414-top 2023-02-13 04:56:21,356 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224-top 2023-02-13 04:56:21,359 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c-top 2023-02-13 04:56:21,359 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/.tmp/cf2/e7bffa8fd1344094b112a00e7f51cc30 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/e7bffa8fd1344094b112a00e7f51cc30 2023-02-13 04:56:21,359 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/.tmp/cf1/6a98b8cd86924f1581aa7c5614b1191d as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/6a98b8cd86924f1581aa7c5614b1191d 2023-02-13 04:56:21,361 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb-top 2023-02-13 04:56:21,364 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3178d3aa6e4f4233a6aa9320e64e5156/cf2 of 3178d3aa6e4f4233a6aa9320e64e5156 into e7bffa8fd1344094b112a00e7f51cc30(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:21,364 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3178d3aa6e4f4233a6aa9320e64e5156: 2023-02-13 04:56:21,364 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156., storeName=3178d3aa6e4f4233a6aa9320e64e5156/cf2, priority=9, startTime=1676264181326; duration=0sec 2023-02-13 04:56:21,364 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,364 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3178d3aa6e4f4233a6aa9320e64e5156/cf1 of 3178d3aa6e4f4233a6aa9320e64e5156 into 6a98b8cd86924f1581aa7c5614b1191d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:21,364 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816-top 2023-02-13 04:56:21,364 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3178d3aa6e4f4233a6aa9320e64e5156: 2023-02-13 04:56:21,364 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156., storeName=3178d3aa6e4f4233a6aa9320e64e5156/cf1, priority=9, startTime=1676264181325; duration=0sec 2023-02-13 04:56:21,364 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,367 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453-top 2023-02-13 04:56:21,367 INFO [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(310): Store=456807945687418d7719c5317d1c58d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:21,367 INFO [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,367 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2 2023-02-13 04:56:21,367 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2 2023-02-13 04:56:21,368 INFO [StoreOpener-456807945687418d7719c5317d1c58d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 456807945687418d7719c5317d1c58d1 columnFamilyName cf2 2023-02-13 04:56:21,378 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435-top 2023-02-13 04:56:21,381 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba-top 2023-02-13 04:56:21,383 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353-top 2023-02-13 04:56:21,386 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167-top 2023-02-13 04:56:21,389 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e-top 2023-02-13 04:56:21,392 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d-top 2023-02-13 04:56:21,395 DEBUG [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(539): loaded hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba-top 2023-02-13 04:56:21,395 INFO [StoreOpener-456807945687418d7719c5317d1c58d1-1] regionserver.HStore(310): Store=456807945687418d7719c5317d1c58d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 04:56:21,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 04:56:21,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 456807945687418d7719c5317d1c58d1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63945767, jitterRate=-0.04713381826877594}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 04:56:21,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 456807945687418d7719c5317d1c58d1: Running coprocessor pre-open hook at 1676264181329Writing region info on filesystem at 1676264181329Initializing all the Stores at 1676264181329Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264181329Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676264181330 (+1 ms)Cleaning up temporary data from old regions at 1676264181397 (+67 ms)Cleaning up detritus from prior splits at 1676264181397Running coprocessor post-open hooks at 1676264181399 (+2 ms)Region opened successfully at 1676264181399 2023-02-13 04:56:21,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1., pid=149, masterSystemTime=1676264181253 2023-02-13 04:56:21,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,401 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:21,401 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 04:56:21,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,401 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,401 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,401 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1912): 456807945687418d7719c5317d1c58d1/cf1 is initiating minor compaction (all files) 2023-02-13 04:56:21,401 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1912): 456807945687418d7719c5317d1c58d1/cf2 is initiating minor compaction (all files) 2023-02-13 04:56:21,401 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 456807945687418d7719c5317d1c58d1/cf1 in testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,401 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 456807945687418d7719c5317d1c58d1/cf2 in testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,401 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652-top] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/.tmp, totalSize=34.9 K 2023-02-13 04:56:21,401 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba-top] into tmpdir=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/.tmp, totalSize=34.9 K 2023-02-13 04:56:21,402 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264177856 2023-02-13 04:56:21,402 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676264177856 2023-02-13 04:56:21,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,402 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264178026 2023-02-13 04:56:21,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,402 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676264178026 2023-02-13 04:56:21,402 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=456807945687418d7719c5317d1c58d1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,402 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264178200 2023-02-13 04:56:21,403 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181402"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676264181402"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676264181402"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676264181402"}]},"ts":"1676264181402"} 2023-02-13 04:56:21,403 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676264178200 2023-02-13 04:56:21,403 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264178371 2023-02-13 04:56:21,403 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676264178371 2023-02-13 04:56:21,403 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676264178549 2023-02-13 04:56:21,403 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676264178549 2023-02-13 04:56:21,404 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676264178718 2023-02-13 04:56:21,404 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676264178718 2023-02-13 04:56:21,404 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] compactions.Compactor(207): Compacting 06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264179703 2023-02-13 04:56:21,404 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] compactions.Compactor(207): Compacting 31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676264179703 2023-02-13 04:56:21,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-13 04:56:21,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 456807945687418d7719c5317d1c58d1, server=jenkins-hbase4.apache.org,33859,1676264072829 in 302 msec 2023-02-13 04:56:21,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-13 04:56:21,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, ASSIGN in 459 msec 2023-02-13 04:56:21,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=714bc3f9fc4c0faad6c7966de7ffb94e, daughterA=3178d3aa6e4f4233a6aa9320e64e5156, daughterB=456807945687418d7719c5317d1c58d1 in 1.6980 sec 2023-02-13 04:56:21,414 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] throttle.PressureAwareThroughputController(145): 456807945687418d7719c5317d1c58d1#cf1#compaction#144 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:21,414 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] throttle.PressureAwareThroughputController(145): 456807945687418d7719c5317d1c58d1#cf2#compaction#145 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 04:56:21,425 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/.tmp/cf2/7854bf17ef50444ca3d0932a1c48a160 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/7854bf17ef50444ca3d0932a1c48a160 2023-02-13 04:56:21,429 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 456807945687418d7719c5317d1c58d1/cf2 of 456807945687418d7719c5317d1c58d1 into 7854bf17ef50444ca3d0932a1c48a160(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:21,429 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 456807945687418d7719c5317d1c58d1: 2023-02-13 04:56:21,429 INFO [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1., storeName=456807945687418d7719c5317d1c58d1/cf2, priority=9, startTime=1676264181401; duration=0sec 2023-02-13 04:56:21,429 DEBUG [RS:1;jenkins-hbase4:33859-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33859] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:50508 deadline: 1676264241460, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e. is not online on jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,570 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:56:21,574 INFO [Listener at localhost/33551] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 04:56:21,576 DEBUG [Listener at localhost/33551] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 04:56:21,592 WARN [Listener at localhost/33551] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 04:56:21,594 INFO [Listener at localhost/33551] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 04:56:21,594 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-13 04:56:21,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 04:56:21,598 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264181597"}]},"ts":"1676264181597"} 2023-02-13 04:56:21,599 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 04:56:21,600 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 04:56:21,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, UNASSIGN}] 2023-02-13 04:56:21,601 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, UNASSIGN 2023-02-13 04:56:21,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=456807945687418d7719c5317d1c58d1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,602 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181602"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264181602"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264181602"}]},"ts":"1676264181602"} 2023-02-13 04:56:21,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 456807945687418d7719c5317d1c58d1, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:21,605 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, UNASSIGN 2023-02-13 04:56:21,605 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=3178d3aa6e4f4233a6aa9320e64e5156, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:21,605 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181605"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676264181605"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676264181605"}]},"ts":"1676264181605"} 2023-02-13 04:56:21,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 3178d3aa6e4f4233a6aa9320e64e5156, server=jenkins-hbase4.apache.org,33859,1676264072829}] 2023-02-13 04:56:21,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 04:56:21,742 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 04:56:21,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 456807945687418d7719c5317d1c58d1, disabling compactions & flushes 2023-02-13 04:56:21,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,823 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/.tmp/cf1/9eb0cba4ed294292ac2a705f50a958ec as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/9eb0cba4ed294292ac2a705f50a958ec 2023-02-13 04:56:21,827 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 456807945687418d7719c5317d1c58d1/cf1 of 456807945687418d7719c5317d1c58d1 into 9eb0cba4ed294292ac2a705f50a958ec(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 04:56:21,827 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 456807945687418d7719c5317d1c58d1: 2023-02-13 04:56:21,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,827 INFO [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1., storeName=456807945687418d7719c5317d1c58d1/cf1, priority=9, startTime=1676264181400; duration=0sec 2023-02-13 04:56:21,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. after waiting 0 ms 2023-02-13 04:56:21,827 DEBUG [RS:1;jenkins-hbase4:33859-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 04:56:21,827 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,828 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652-top] to archive 2023-02-13 04:56:21,828 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:21,829 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,830 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,831 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,832 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,833 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,834 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,836 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,837 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167-top, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba-top] to archive 2023-02-13 04:56:21,837 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:21,839 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,840 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,841 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,843 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,844 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,845 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,846 DEBUG [StoreCloser-testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:56:21,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:21,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1. 2023-02-13 04:56:21,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 456807945687418d7719c5317d1c58d1: Waiting for close lock at 1676264181755Running coprocessor pre-close hooks at 1676264181755Disabling compacts and flushes for region at 1676264181755Disabling writes for close at 1676264181827 (+72 ms)Writing region close event to WAL at 1676264181847 (+20 ms)Running coprocessor post-close hooks at 1676264181849 (+2 ms)Closed at 1676264181850 (+1 ms) 2023-02-13 04:56:21,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3178d3aa6e4f4233a6aa9320e64e5156, disabling compactions & flushes 2023-02-13 04:56:21,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. after waiting 0 ms 2023-02-13 04:56:21,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,852 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=456807945687418d7719c5317d1c58d1, regionState=CLOSED 2023-02-13 04:56:21,852 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181852"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264181852"}]},"ts":"1676264181852"} 2023-02-13 04:56:21,853 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652-bottom] to archive 2023-02-13 04:56:21,854 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:21,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 04:56:21,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 456807945687418d7719c5317d1c58d1, server=jenkins-hbase4.apache.org,33859,1676264072829 in 251 msec 2023-02-13 04:56:21,855 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/0a2017bd3763487981d2d4a944afe414.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=456807945687418d7719c5317d1c58d1, UNASSIGN in 254 msec 2023-02-13 04:56:21,857 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/799becb3b2904cb4be3815a38154facb.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,858 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/43801071b8bf40599c613a1d39886224.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,859 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/f06f89a77ad7447587594280ad7dc453.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,860 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/887a50d0c36142e6925ea0357a455816.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,861 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/650ed5d5a6cc4516ab9acdeebb47d04c.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,862 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/06cfff59c698402299e9e01846cee652.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,864 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167-bottom, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e->hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba-bottom] to archive 2023-02-13 04:56:21,864 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 04:56:21,865 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/9fa45b953da34be4848951d53fd0b71e.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,866 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/1eb1baa91f964285900c7cde94bae435.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,867 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/ec7b80b6c87e4947a6729118372c53ba.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,868 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/4db7c7a4ca794d18888df2951db6b353.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,869 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/de7538ae9d62408c8d38e5953bbd6d6d.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,869 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/95e62fee42144a65b14141c5f3425167.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,870 DEBUG [StoreCloser-testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/31a8f2b8e62d4221bf44b3c32f4314ba.714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 04:56:21,874 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:21,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156. 2023-02-13 04:56:21,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3178d3aa6e4f4233a6aa9320e64e5156: Waiting for close lock at 1676264181852Running coprocessor pre-close hooks at 1676264181852Disabling compacts and flushes for region at 1676264181852Disabling writes for close at 1676264181852Writing region close event to WAL at 1676264181871 (+19 ms)Running coprocessor post-close hooks at 1676264181874 (+3 ms)Closed at 1676264181874 2023-02-13 04:56:21,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,876 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=3178d3aa6e4f4233a6aa9320e64e5156, regionState=CLOSED 2023-02-13 04:56:21,876 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676264181876"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676264181876"}]},"ts":"1676264181876"} 2023-02-13 04:56:21,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 04:56:21,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 3178d3aa6e4f4233a6aa9320e64e5156, server=jenkins-hbase4.apache.org,33859,1676264072829 in 271 msec 2023-02-13 04:56:21,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-13 04:56:21,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3178d3aa6e4f4233a6aa9320e64e5156, UNASSIGN in 277 msec 2023-02-13 04:56:21,879 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676264181879"}]},"ts":"1676264181879"} 2023-02-13 04:56:21,880 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 04:56:21,881 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 04:56:21,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 288 msec 2023-02-13 04:56:21,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 04:56:21,899 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 04:56:21,899 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-13 04:56:21,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,902 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 04:56:21,905 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,905 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,905 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,907 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/recovered.edits] 2023-02-13 04:56:21,907 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/recovered.edits] 2023-02-13 04:56:21,907 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2, FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/recovered.edits] 2023-02-13 04:56:21,909 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/6a98b8cd86924f1581aa7c5614b1191d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf1/6a98b8cd86924f1581aa7c5614b1191d 2023-02-13 04:56:21,909 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/9eb0cba4ed294292ac2a705f50a958ec to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf1/9eb0cba4ed294292ac2a705f50a958ec 2023-02-13 04:56:21,910 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/06cfff59c698402299e9e01846cee652 2023-02-13 04:56:21,911 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/0a2017bd3763487981d2d4a944afe414 2023-02-13 04:56:21,911 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/e7bffa8fd1344094b112a00e7f51cc30 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/cf2/e7bffa8fd1344094b112a00e7f51cc30 2023-02-13 04:56:21,911 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/7854bf17ef50444ca3d0932a1c48a160 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/cf2/7854bf17ef50444ca3d0932a1c48a160 2023-02-13 04:56:21,912 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/43801071b8bf40599c613a1d39886224 2023-02-13 04:56:21,914 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/650ed5d5a6cc4516ab9acdeebb47d04c 2023-02-13 04:56:21,914 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156/recovered.edits/90.seqid 2023-02-13 04:56:21,914 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/recovered.edits/90.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1/recovered.edits/90.seqid 2023-02-13 04:56:21,914 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/3178d3aa6e4f4233a6aa9320e64e5156 2023-02-13 04:56:21,914 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/456807945687418d7719c5317d1c58d1 2023-02-13 04:56:21,915 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/799becb3b2904cb4be3815a38154facb 2023-02-13 04:56:21,916 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/887a50d0c36142e6925ea0357a455816 2023-02-13 04:56:21,917 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf1/f06f89a77ad7447587594280ad7dc453 2023-02-13 04:56:21,919 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/1eb1baa91f964285900c7cde94bae435 2023-02-13 04:56:21,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/31a8f2b8e62d4221bf44b3c32f4314ba 2023-02-13 04:56:21,921 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/4db7c7a4ca794d18888df2951db6b353 2023-02-13 04:56:21,921 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167 to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/95e62fee42144a65b14141c5f3425167 2023-02-13 04:56:21,922 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/9fa45b953da34be4848951d53fd0b71e 2023-02-13 04:56:21,923 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/de7538ae9d62408c8d38e5953bbd6d6d 2023-02-13 04:56:21,924 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/cf2/ec7b80b6c87e4947a6729118372c53ba 2023-02-13 04:56:21,926 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/recovered.edits/85.seqid to hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/archive/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e/recovered.edits/85.seqid 2023-02-13 04:56:21,926 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/.tmp/data/default/testSplitTableRegion/714bc3f9fc4c0faad6c7966de7ffb94e 2023-02-13 04:56:21,926 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 04:56:21,928 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,929 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 04:56:21,931 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 04:56:21,931 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,932 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 04:56:21,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264181932"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:21,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264181932"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:21,932 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676264181932"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:21,933 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 04:56:21,933 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 714bc3f9fc4c0faad6c7966de7ffb94e, NAME => 'testSplitTableRegion,,1676264177353.714bc3f9fc4c0faad6c7966de7ffb94e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3178d3aa6e4f4233a6aa9320e64e5156, NAME => 'testSplitTableRegion,,1676264179708.3178d3aa6e4f4233a6aa9320e64e5156.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 456807945687418d7719c5317d1c58d1, NAME => 'testSplitTableRegion,41,1676264179708.456807945687418d7719c5317d1c58d1.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 04:56:21,933 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 04:56:21,933 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676264181933"}]},"ts":"9223372036854775807"} 2023-02-13 04:56:21,934 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 04:56:21,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 04:56:21,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 36 msec 2023-02-13 04:56:22,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43801] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 04:56:22,003 INFO [Listener at localhost/33551] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 04:56:22,017 INFO [Listener at localhost/33551] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374 (was 375), OpenFileDescriptor=827 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=114 (was 114), ProcessCount=168 (was 168), AvailableMemoryMB=3411 (was 3478) 2023-02-13 04:56:22,017 INFO [Listener at localhost/33551] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 04:56:22,017 INFO [Listener at localhost/33551] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 04:56:22,021 DEBUG [Listener at localhost/33551] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c8d5aec to 127.0.0.1:55056 2023-02-13 04:56:22,021 DEBUG [Listener at localhost/33551] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,022 DEBUG [Listener at localhost/33551] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 04:56:22,022 DEBUG [Listener at localhost/33551] util.JVMClusterUtil(257): Found active master hash=497556708, stopped=false 2023-02-13 04:56:22,022 INFO [Listener at localhost/33551] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:56:22,029 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 04:56:22,029 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 04:56:22,029 INFO [Listener at localhost/33551] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 04:56:22,030 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 04:56:22,030 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 04:56:22,030 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:56:22,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:56:22,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:56:22,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:56:22,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 04:56:22,032 DEBUG [Listener at localhost/33551] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x02cce10f to 127.0.0.1:55056 2023-02-13 04:56:22,032 DEBUG [Listener at localhost/33551] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,032 INFO [Listener at localhost/33551] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46601,1676264072787' ***** 2023-02-13 04:56:22,032 INFO [Listener at localhost/33551] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 04:56:22,032 INFO [RS:0;jenkins-hbase4:46601] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 04:56:22,033 INFO [RS:0;jenkins-hbase4:46601] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 04:56:22,033 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 04:56:22,033 INFO [RS:0;jenkins-hbase4:46601] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 04:56:22,033 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:22,033 DEBUG [RS:0;jenkins-hbase4:46601] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x15e8e6b1 to 127.0.0.1:55056 2023-02-13 04:56:22,033 DEBUG [RS:0;jenkins-hbase4:46601] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,033 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46601,1676264072787; all regions closed. 2023-02-13 04:56:22,033 INFO [Listener at localhost/33551] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33859,1676264072829' ***** 2023-02-13 04:56:22,033 INFO [Listener at localhost/33551] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 04:56:22,034 INFO [Listener at localhost/33551] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36009,1676264072859' ***** 2023-02-13 04:56:22,034 INFO [Listener at localhost/33551] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 04:56:22,034 INFO [RS:1;jenkins-hbase4:33859] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 04:56:22,034 INFO [RS:1;jenkins-hbase4:33859] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 04:56:22,034 INFO [RS:1;jenkins-hbase4:33859] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 04:56:22,034 INFO [RS:2;jenkins-hbase4:36009] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 04:56:22,034 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:22,034 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 04:56:22,034 INFO [RS:2;jenkins-hbase4:36009] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 04:56:22,034 INFO [RS:2;jenkins-hbase4:36009] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 04:56:22,034 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(3304): Received CLOSE for dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:56:22,035 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:22,035 DEBUG [RS:2;jenkins-hbase4:36009] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1c9d1aed to 127.0.0.1:55056 2023-02-13 04:56:22,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing dc413a9db9894ddca28879b9c07ec853, disabling compactions & flushes 2023-02-13 04:56:22,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:56:22,034 DEBUG [RS:1;jenkins-hbase4:33859] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5cf9bd84 to 127.0.0.1:55056 2023-02-13 04:56:22,034 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 04:56:22,035 DEBUG [RS:1;jenkins-hbase4:33859] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,035 INFO [RS:1;jenkins-hbase4:33859] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 04:56:22,035 INFO [RS:1;jenkins-hbase4:33859] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 04:56:22,035 INFO [RS:1;jenkins-hbase4:33859] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 04:56:22,035 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 04:56:22,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:56:22,035 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 04:56:22,035 DEBUG [RS:2;jenkins-hbase4:36009] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,036 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 04:56:22,036 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 04:56:22,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. after waiting 0 ms 2023-02-13 04:56:22,036 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 04:56:22,036 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 04:56:22,036 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 04:56:22,036 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:56:22,036 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 04:56:22,036 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1479): Online Regions={dc413a9db9894ddca28879b9c07ec853=hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853.} 2023-02-13 04:56:22,036 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 04:56:22,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing dc413a9db9894ddca28879b9c07ec853 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 04:56:22,036 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-13 04:56:22,205 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 04:56:22,205 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 04:56:22,205 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 04:56:22,205 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 04:56:22,205 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 04:56:22,206 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 04:56:22,206 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 04:56:22,205 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 04:56:22,205 DEBUG [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1505): Waiting on dc413a9db9894ddca28879b9c07ec853 2023-02-13 04:56:22,205 DEBUG [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 04:56:22,206 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 04:56:22,216 DEBUG [RS:0;jenkins-hbase4:46601] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs 2023-02-13 04:56:22,216 INFO [RS:0;jenkins-hbase4:46601] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46601%2C1676264072787:(num 1676264074316) 2023-02-13 04:56:22,216 DEBUG [RS:0;jenkins-hbase4:46601] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,217 INFO [RS:0;jenkins-hbase4:46601] regionserver.LeaseManager(133): Closed leases 2023-02-13 04:56:22,217 INFO [RS:0;jenkins-hbase4:46601] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 04:56:22,224 INFO [RS:0;jenkins-hbase4:46601] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 04:56:22,224 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 04:56:22,224 INFO [RS:0;jenkins-hbase4:46601] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 04:56:22,224 INFO [RS:0;jenkins-hbase4:46601] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 04:56:22,225 INFO [RS:0;jenkins-hbase4:46601] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46601 2023-02-13 04:56:22,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/.tmp/info/57c3b0e75b9e49379f31df8ee96fdaed 2023-02-13 04:56:22,231 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/.tmp/info/9667a1688043428fb57687679d2b7bf4 2023-02-13 04:56:22,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/.tmp/info/57c3b0e75b9e49379f31df8ee96fdaed as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/info/57c3b0e75b9e49379f31df8ee96fdaed 2023-02-13 04:56:22,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:22,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:56:22,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:56:22,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:22,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46601,1676264072787 2023-02-13 04:56:22,232 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73ecba88 rejected from java.util.concurrent.ThreadPoolExecutor@77d3202d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,232 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:56:22,233 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:56:22,233 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@15877417 rejected from java.util.concurrent.ThreadPoolExecutor@77d3202d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,233 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46601,1676264072787] 2023-02-13 04:56:22,233 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46601,1676264072787; numProcessing=1 2023-02-13 04:56:22,234 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9667a1688043428fb57687679d2b7bf4 2023-02-13 04:56:22,234 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46601,1676264072787 already deleted, retry=false 2023-02-13 04:56:22,235 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46601,1676264072787 expired; onlineServers=2 2023-02-13 04:56:22,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/info/57c3b0e75b9e49379f31df8ee96fdaed, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 04:56:22,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for dc413a9db9894ddca28879b9c07ec853 in 201ms, sequenceid=6, compaction requested=false 2023-02-13 04:56:22,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/namespace/dc413a9db9894ddca28879b9c07ec853/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 04:56:22,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:22,245 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:56:22,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for dc413a9db9894ddca28879b9c07ec853: Waiting for close lock at 1676264182035Running coprocessor pre-close hooks at 1676264182035Disabling compacts and flushes for region at 1676264182035Disabling writes for close at 1676264182036 (+1 ms)Obtaining lock to block concurrent updates at 1676264182036Preparing flush snapshotting stores in dc413a9db9894ddca28879b9c07ec853 at 1676264182036Finished memstore snapshotting hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676264182036Flushing stores of hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. at 1676264182037 (+1 ms)Flushing dc413a9db9894ddca28879b9c07ec853/info: creating writer at 1676264182037Flushing dc413a9db9894ddca28879b9c07ec853/info: appending metadata at 1676264182208 (+171 ms)Flushing dc413a9db9894ddca28879b9c07ec853/info: closing flushed file at 1676264182209 (+1 ms)Flushing dc413a9db9894ddca28879b9c07ec853/info: reopening flushed file at 1676264182232 (+23 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for dc413a9db9894ddca28879b9c07ec853 in 201ms, sequenceid=6, compaction requested=false at 1676264182237 (+5 ms)Writing region close event to WAL at 1676264182238 (+1 ms)Running coprocessor post-close hooks at 1676264182244 (+6 ms)Closed at 1676264182245 (+1 ms) 2023-02-13 04:56:22,245 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676264074711.dc413a9db9894ddca28879b9c07ec853. 2023-02-13 04:56:22,247 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/.tmp/rep_barrier/4fa3ce35fabb4a9ba370ab1eec4b391b 2023-02-13 04:56:22,250 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4fa3ce35fabb4a9ba370ab1eec4b391b 2023-02-13 04:56:22,263 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/.tmp/table/92cbf7de89ac4aef9e27626d116bf3e0 2023-02-13 04:56:22,266 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92cbf7de89ac4aef9e27626d116bf3e0 2023-02-13 04:56:22,266 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/.tmp/info/9667a1688043428fb57687679d2b7bf4 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/info/9667a1688043428fb57687679d2b7bf4 2023-02-13 04:56:22,269 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9667a1688043428fb57687679d2b7bf4 2023-02-13 04:56:22,269 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/info/9667a1688043428fb57687679d2b7bf4, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 04:56:22,270 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/.tmp/rep_barrier/4fa3ce35fabb4a9ba370ab1eec4b391b as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/rep_barrier/4fa3ce35fabb4a9ba370ab1eec4b391b 2023-02-13 04:56:22,273 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4fa3ce35fabb4a9ba370ab1eec4b391b 2023-02-13 04:56:22,273 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/rep_barrier/4fa3ce35fabb4a9ba370ab1eec4b391b, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 04:56:22,273 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/.tmp/table/92cbf7de89ac4aef9e27626d116bf3e0 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/table/92cbf7de89ac4aef9e27626d116bf3e0 2023-02-13 04:56:22,276 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92cbf7de89ac4aef9e27626d116bf3e0 2023-02-13 04:56:22,276 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/table/92cbf7de89ac4aef9e27626d116bf3e0, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 04:56:22,277 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 241ms, sequenceid=210, compaction requested=false 2023-02-13 04:56:22,281 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 04:56:22,281 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 04:56:22,281 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 04:56:22,282 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 04:56:22,282 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676264182036Running coprocessor pre-close hooks at 1676264182036Disabling compacts and flushes for region at 1676264182036Disabling writes for close at 1676264182036Obtaining lock to block concurrent updates at 1676264182036Preparing flush snapshotting stores in 1588230740 at 1676264182036Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676264182036Flushing stores of hbase:meta,,1.1588230740 at 1676264182037 (+1 ms)Flushing 1588230740/info: creating writer at 1676264182037Flushing 1588230740/info: appending metadata at 1676264182212 (+175 ms)Flushing 1588230740/info: closing flushed file at 1676264182212Flushing 1588230740/rep_barrier: creating writer at 1676264182235 (+23 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676264182236 (+1 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676264182236Flushing 1588230740/table: creating writer at 1676264182250 (+14 ms)Flushing 1588230740/table: appending metadata at 1676264182252 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676264182252Flushing 1588230740/info: reopening flushed file at 1676264182267 (+15 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676264182270 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1676264182274 (+4 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 241ms, sequenceid=210, compaction requested=false at 1676264182277 (+3 ms)Writing region close event to WAL at 1676264182278 (+1 ms)Running coprocessor post-close hooks at 1676264182281 (+3 ms)Closed at 1676264182282 (+1 ms) 2023-02-13 04:56:22,282 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 04:56:22,334 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,334 INFO [RS:0;jenkins-hbase4:46601] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46601,1676264072787; zookeeper connection closed. 2023-02-13 04:56:22,334 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6d953ed5 rejected from java.util.concurrent.ThreadPoolExecutor@77d3202d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,334 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:46601-0x1004c07c8780001, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,334 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7c77ae78 rejected from java.util.concurrent.ThreadPoolExecutor@77d3202d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,334 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@12b1b7d1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@12b1b7d1 2023-02-13 04:56:22,406 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36009,1676264072859; all regions closed. 2023-02-13 04:56:22,407 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33859,1676264072829; all regions closed. 2023-02-13 04:56:22,413 DEBUG [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs 2023-02-13 04:56:22,414 DEBUG [RS:2;jenkins-hbase4:36009] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs 2023-02-13 04:56:22,414 INFO [RS:2;jenkins-hbase4:36009] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36009%2C1676264072859:(num 1676264074316) 2023-02-13 04:56:22,414 INFO [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33859%2C1676264072829.meta:.meta(num 1676264074433) 2023-02-13 04:56:22,414 DEBUG [RS:2;jenkins-hbase4:36009] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,414 INFO [RS:2;jenkins-hbase4:36009] regionserver.LeaseManager(133): Closed leases 2023-02-13 04:56:22,414 INFO [RS:2;jenkins-hbase4:36009] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 04:56:22,415 INFO [RS:2;jenkins-hbase4:36009] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 04:56:22,415 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 04:56:22,415 INFO [RS:2;jenkins-hbase4:36009] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 04:56:22,415 INFO [RS:2;jenkins-hbase4:36009] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 04:56:22,416 INFO [RS:2;jenkins-hbase4:36009] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36009 2023-02-13 04:56:22,419 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:22,419 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:56:22,419 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2b8f3ff9 rejected from java.util.concurrent.ThreadPoolExecutor@2fb0be50[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,419 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36009,1676264072859 2023-02-13 04:56:22,419 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36009,1676264072859] 2023-02-13 04:56:22,419 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36009,1676264072859; numProcessing=2 2023-02-13 04:56:22,420 DEBUG [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/oldWALs 2023-02-13 04:56:22,420 INFO [RS:1;jenkins-hbase4:33859] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33859%2C1676264072829:(num 1676264074316) 2023-02-13 04:56:22,420 DEBUG [RS:1;jenkins-hbase4:33859] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,420 INFO [RS:1;jenkins-hbase4:33859] regionserver.LeaseManager(133): Closed leases 2023-02-13 04:56:22,420 INFO [RS:1;jenkins-hbase4:33859] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 04:56:22,420 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 04:56:22,421 INFO [RS:1;jenkins-hbase4:33859] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33859 2023-02-13 04:56:22,421 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36009,1676264072859 already deleted, retry=false 2023-02-13 04:56:22,421 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36009,1676264072859 expired; onlineServers=1 2023-02-13 04:56:22,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33859,1676264072829 2023-02-13 04:56:22,422 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 04:56:22,422 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@640e5b64 rejected from java.util.concurrent.ThreadPoolExecutor@6031c33d[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,424 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33859,1676264072829] 2023-02-13 04:56:22,424 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33859,1676264072829; numProcessing=3 2023-02-13 04:56:22,425 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33859,1676264072829 already deleted, retry=false 2023-02-13 04:56:22,425 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33859,1676264072829 expired; onlineServers=0 2023-02-13 04:56:22,425 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43801,1676264071669' ***** 2023-02-13 04:56:22,425 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 04:56:22,426 DEBUG [M:0;jenkins-hbase4:43801] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c9516df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 04:56:22,426 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43801,1676264071669 2023-02-13 04:56:22,426 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43801,1676264071669; all regions closed. 2023-02-13 04:56:22,426 DEBUG [M:0;jenkins-hbase4:43801] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 04:56:22,426 DEBUG [M:0;jenkins-hbase4:43801] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 04:56:22,426 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 04:56:22,426 DEBUG [M:0;jenkins-hbase4:43801] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 04:56:22,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264073946] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676264073946,5,FailOnTimeoutGroup] 2023-02-13 04:56:22,426 INFO [M:0;jenkins-hbase4:43801] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 04:56:22,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264073947] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676264073947,5,FailOnTimeoutGroup] 2023-02-13 04:56:22,426 INFO [M:0;jenkins-hbase4:43801] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 04:56:22,427 INFO [M:0;jenkins-hbase4:43801] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 04:56:22,427 DEBUG [M:0;jenkins-hbase4:43801] master.HMaster(1502): Stopping service threads 2023-02-13 04:56:22,427 INFO [M:0;jenkins-hbase4:43801] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 04:56:22,427 ERROR [M:0;jenkins-hbase4:43801] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 04:56:22,427 INFO [M:0;jenkins-hbase4:43801] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 04:56:22,427 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 04:56:22,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 04:56:22,427 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 04:56:22,428 DEBUG [M:0;jenkins-hbase4:43801] zookeeper.ZKUtil(398): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 04:56:22,428 WARN [M:0;jenkins-hbase4:43801] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 04:56:22,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 04:56:22,428 INFO [M:0;jenkins-hbase4:43801] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 04:56:22,428 INFO [M:0;jenkins-hbase4:43801] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 04:56:22,428 DEBUG [M:0;jenkins-hbase4:43801] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 04:56:22,428 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:56:22,428 DEBUG [M:0;jenkins-hbase4:43801] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:56:22,428 DEBUG [M:0;jenkins-hbase4:43801] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 04:56:22,428 DEBUG [M:0;jenkins-hbase4:43801] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:56:22,428 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.14 KB heapSize=719.05 KB 2023-02-13 04:56:22,442 INFO [M:0;jenkins-hbase4:43801] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.14 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/af3d31ba010d4dad9456fad67c7c9707 2023-02-13 04:56:22,445 INFO [M:0;jenkins-hbase4:43801] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af3d31ba010d4dad9456fad67c7c9707 2023-02-13 04:56:22,446 DEBUG [M:0;jenkins-hbase4:43801] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/af3d31ba010d4dad9456fad67c7c9707 as hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/af3d31ba010d4dad9456fad67c7c9707 2023-02-13 04:56:22,449 INFO [M:0;jenkins-hbase4:43801] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for af3d31ba010d4dad9456fad67c7c9707 2023-02-13 04:56:22,449 INFO [M:0;jenkins-hbase4:43801] regionserver.HStore(1080): Added hdfs://localhost:44231/user/jenkins/test-data/6530fdf5-f415-2f37-6b28-e50630fd00fb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/af3d31ba010d4dad9456fad67c7c9707, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 04:56:22,450 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegion(2947): Finished flush of dataSize ~608.14 KB/622739, heapSize ~719.04 KB/736296, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=1282, compaction requested=false 2023-02-13 04:56:22,451 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 04:56:22,451 DEBUG [M:0;jenkins-hbase4:43801] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676264182428Disabling compacts and flushes for region at 1676264182428Disabling writes for close at 1676264182428Obtaining lock to block concurrent updates at 1676264182428Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676264182428Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622739, getHeapSize=736296, getOffHeapSize=0, getCellsCount=1499 at 1676264182428Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676264182429 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676264182429Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676264182432 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676264182432Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676264182447 (+15 ms)Finished flush of dataSize ~608.14 KB/622739, heapSize ~719.04 KB/736296, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 22ms, sequenceid=1282, compaction requested=false at 1676264182450 (+3 ms)Writing region close event to WAL at 1676264182451 (+1 ms)Closed at 1676264182451 2023-02-13 04:56:22,454 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 04:56:22,454 INFO [M:0;jenkins-hbase4:43801] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 04:56:22,455 INFO [M:0;jenkins-hbase4:43801] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43801 2023-02-13 04:56:22,456 DEBUG [M:0;jenkins-hbase4:43801] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43801,1676264071669 already deleted, retry=false 2023-02-13 04:56:22,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,521 INFO [RS:2;jenkins-hbase4:36009] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36009,1676264072859; zookeeper connection closed. 2023-02-13 04:56:22,521 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@9b00c61 rejected from java.util.concurrent.ThreadPoolExecutor@2fb0be50[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,521 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:36009-0x1004c07c8780003, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,521 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5d5cc5a1] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5d5cc5a1 2023-02-13 04:56:22,521 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@717afd45 rejected from java.util.concurrent.ThreadPoolExecutor@2fb0be50[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,621 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,621 INFO [M:0;jenkins-hbase4:43801] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43801,1676264071669; zookeeper connection closed. 2023-02-13 04:56:22,621 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@72fff2f8 rejected from java.util.concurrent.ThreadPoolExecutor@515cba6c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,622 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): master:43801-0x1004c07c8780000, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,622 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73063a74 rejected from java.util.concurrent.ThreadPoolExecutor@515cba6c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,721 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,721 INFO [RS:1;jenkins-hbase4:33859] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33859,1676264072829; zookeeper connection closed. 2023-02-13 04:56:22,722 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@29f0e74 rejected from java.util.concurrent.ThreadPoolExecutor@6031c33d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,722 DEBUG [Listener at localhost/33551-EventThread] zookeeper.ZKWatcher(600): regionserver:33859-0x1004c07c8780002, quorum=127.0.0.1:55056, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 04:56:22,722 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5ea9de0] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5ea9de0 2023-02-13 04:56:22,722 ERROR [Listener at localhost/33551-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7a9ab1b5 rejected from java.util.concurrent.ThreadPoolExecutor@6031c33d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 04:56:22,722 INFO [Listener at localhost/33551] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 04:56:22,723 WARN [Listener at localhost/33551] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 04:56:22,725 INFO [Listener at localhost/33551] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 04:56:22,827 WARN [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 04:56:22,827 WARN [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid 82110bba-6b14-4557-aa3b-b536cbd539dd) service to localhost/127.0.0.1:44231 2023-02-13 04:56:22,830 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data5/current/BP-1205038681-172.31.14.131-1676264068588] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 04:56:22,830 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data6/current/BP-1205038681-172.31.14.131-1676264068588] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 04:56:22,831 WARN [Listener at localhost/33551] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 04:56:22,833 INFO [Listener at localhost/33551] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 04:56:22,938 WARN [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid 2af29379-a15b-4462-998d-be0242ce1402) service to localhost/127.0.0.1:44231 2023-02-13 04:56:22,939 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data3/current/BP-1205038681-172.31.14.131-1676264068588] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 04:56:22,939 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data4/current/BP-1205038681-172.31.14.131-1676264068588] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 04:56:22,940 WARN [Listener at localhost/33551] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 04:56:22,947 INFO [Listener at localhost/33551] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 04:56:23,053 WARN [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 04:56:23,053 WARN [BP-1205038681-172.31.14.131-1676264068588 heartbeating to localhost/127.0.0.1:44231] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1205038681-172.31.14.131-1676264068588 (Datanode Uuid a51c43df-067e-41c1-a591-2b7ca0d868cc) service to localhost/127.0.0.1:44231 2023-02-13 04:56:23,055 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data1/current/BP-1205038681-172.31.14.131-1676264068588] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 04:56:23,056 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/25671feb-94f2-6bda-1a77-c78ba265b655/cluster_831170e5-a05b-785f-64f1-0ab9f14ae8ea/dfs/data/data2/current/BP-1205038681-172.31.14.131-1676264068588] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 04:56:23,077 INFO [Listener at localhost/33551] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 04:56:23,264 INFO [Listener at localhost/33551] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 04:56:23,499 INFO [Listener at localhost/33551] hbase.HBaseTestingUtility(1293): Minicluster is down